AFM scanning image processing method based on alternating Kalman filtering and edge operator

Abstract

The invention belongs to the field of AFM scanning imaging restoration, and particularly relates to an AFM scanning image processing method based on alternate Kalman filtering and an edge operator, which comprises the following steps: S1, carrying out graying processing on an image obtained by AFM scanning, and facilitating subsequent processing operation while reducing data volume; and S2, compressing the grayed image by adopting an image compression algorithm based on a generative adversarial network so as to reduce the data dimension and further reduce the data volume required to be processed by alternate Kalman column-row filtering. S3, performing Kalman column and row filtering on the grayed image, and improving the quality of the image through iteration so as to realize high-precision AFM image processing; and S4, using a log edge detection operator to carry out edge feature extraction on the image obtained after alternate Kalman filtering processing, and simplifying the complexity in the image. According to the method, the algorithm has a certain automation characteristic, and filtering and edge detection steps are fused, so that the requirement of manually adjusting parameters is reduced, and the processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of image restoration, and specifically to an AFM scanning image processing method based on alternating Kalman filtering and an edge operator. Background Art

[0002] Target Tracking Technology Atomic Force Microscope (AFM) is an important nanoscale imaging technology, which is widely used in materials science, biology, semiconductor manufacturing and other fields. AFM obtains surface morphology information through the interaction force between the probe and the sample surface, and can achieve high-resolution imaging. However, the original images obtained by AFM are often affected by many factors, resulting in poor image quality. Therefore, image processing is necessary, which can significantly improve the usability of images and the accuracy of analysis.

[0003] First, there is often noise in AFM images. Noise may come from a variety of factors, including environmental vibrations, electronic noise, and the interaction between the probe and the sample. These noises will interfere with the extraction of real signals and affect the analysis of the surface characteristics of the sample. By applying image processing techniques, such as low-pass filtering and Gaussian filtering, these random noises can be effectively removed, thereby improving the clarity and readability of the image. Noise removal not only helps to improve image quality, but also improves the accuracy of subsequent analysis.

[0004] In summary, image processing of atomic force microscopy is a key step to improve image quality and analysis accuracy. Through techniques such as noise removal, artifact correction, and contrast enhancement, researchers can better understand the microscopic properties of samples. This not only helps to promote the development of basic scientific research, but also provides important support for applications in fields such as materials science, nanotechnology, and biomedicine. Therefore, AFM image processing plays an indispensable role in modern scientific research and deserves further exploration and development. Summary of the invention

[0005] 1. Technical issues to be solved

[0006] In view of the deficiencies in the prior art, the present invention provides an AFM output image processing method based on alternating Kalman filtering and edge operator, which solves the problems raised in the above background technology.

[0007] (II) Technical solution

[0008] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical scheme: an AFM scanning image processing method based on alternating Kalman filtering and edge operator, comprising the following steps:

[0009] S1. Grayscale the image obtained by AFM scanning to reduce the amount of data and facilitate subsequent processing operations;

[0010] S2. The grayscale image is compressed using an image compression algorithm based on a generative adversarial network to reduce the data dimension and further reduce the amount of data that needs to be processed by the alternating Kalman column-row filtering.

[0011] S3, performing Kalman column-row filtering on the grayscale image, improving the image quality through iteration, so as to achieve high-precision AFM image processing;

[0012] S4, using the log edge detection operator to extract edge features from the image obtained after the alternating Kalman filter processing, simplifying the complexity in the image and extracting useful structural information;

[0013] Furthermore, in S1, the target image is grayed using the formula:

[0014] I gray (x,y)=0.2989·R(x,y)+0.5870·G(x,y)+0.1140·B(x,y)

[0015] Among them, R, G, B are the red, green and blue channel values ​​of the color image at the corresponding position, I gray (x, y) is the pixel value of the grayscale image at position (x, y). Grayscale processing can simplify image data so that subsequent processing algorithms (such as filtering and edge detection) only need to process one channel, reducing computational complexity. In order to further improve the real-time performance of the system, the image needs to be further compressed.

[0016] Furthermore, the grayscale processed image is compressed according to S2, and the grayscale processed image is compressed using an image compression algorithm based on a generative adversarial network:

[0017] First, train GAN, and the loss function is:

[0018]

[0019] Where G is the generator, D is the discriminator, and p data is the true image distribution, p z is the potential space distribution, x is the real input image, which comes from p data , z is distributed from the latent space p z The sampled noise vector, L, is the loss between the generator and the discriminator, with the goal of minimizing the probability that a fake image generated by the generator is correctly identified as fake by the discriminator during training. After training, the generator is used to compress the AFM image.

[0020] Furthermore, according to S3, data is acquired from the grayscale processed image, and the image features are modeled as states to facilitate subsequent prediction and updating. Prediction and updating are the two main steps of alternating Kalman column-row filtering, and the formula used is as follows:

[0021] Predict current status:

[0022]

[0023] Update the covariance matrix:

[0024] P j,k|k-1 =FP j,k-1|k-1 F T +Q

[0025] Calculate the Kalman gain:

[0026] K j,k =P j,k|k-1 H T (HP j,k|k-1 H T +R) -1

[0027] To predict or update the state, P s,k|k-1 To predict or update the covariance, the predicted value of the uncertainty of the current state, K s,k Kalman gain is used to balance the weight between the predicted value and the observed value, which is equivalent to the weight given to the observed data when updating the state. A larger value means a higher degree of trust in the observation. Kalman gain determines how to balance the importance of predicted and observed data and is an important indicator for updating. Alternating Kalman column-row filtering is based on the continuous iteration of prediction and update steps to gradually improve the quality of image restoration and ultimately achieve high-precision AFM image processing.

[0028] Furthermore, according to S4, the edge feature of the image is extracted using the Log edge detection operator, and the formula is as follows:

[0029]

[0030] x and y are the relative positions of the current pixel in the image coordinate system. x represents the horizontal displacement and y represents the vertical displacement. In the convolution operation, these two parameters help determine the position of the filter application. σ is the standard deviation of Gaussian smoothing, which controls the width of the Gaussian function. It affects the shape and size of the filter. A large σ value will result in a wider smoothing effect, which helps to remove high-frequency noise, while a smaller σ value can capture more subtle edge features. This term is a normalization factor, which is used to ensure that the total energy (or intensity) of the operator remains consistent during integration, so that the output of the operator is not affected by the size of σ, which makes the edge detection result more stable. This part is a directional information, which means that the farther away from the center point, the smaller the impact. It forms positive and negative features near the edge. By adjusting this value to strengthen the edge contrast, it helps to distinguish the positive and negative changes in the edge position and enhance the edge detection effect. It is a component of the Gaussian function, which makes the weight of pixels farther away from the center point lower, and the shape presents circular symmetry, which conforms to the Gaussian distribution characteristics. Through this item, the Log operator achieves a good local smoothing effect, while producing a strong response at the edge position, effectively capturing edge information.

[0031] (III) Beneficial effects

[0032] Compared with the prior art, the present invention provides an AFM scanning image processing method based on alternating Kalman filtering and edge operator, which has the following beneficial effects:

[0033] The algorithm of the present invention has certain automation characteristics and integrates the filtering and edge detection steps, thereby reducing the need for manual parameter adjustment and improving processing efficiency.

[0034] The present invention improves the adaptability and processing capability of two-dimensional images by extending the alternating Kalman column-row filtering to alternately process columns and rows based on the processing mode of traditional Kalman filtering. This method can better capture the spatial features in the image and reduce the directional influence that may be encountered in the processing process of traditional methods.

[0035] The present invention improves the accuracy of feature extraction. After filtering and edge detection fusion processing, the feature extraction result is more accurate and the extraction error is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The following is the overall flow chart;

[0037] Figure 2 It is the flow chart of alternating Kalman column-row filtering;

[0038] Figure 3 This is the AFM image scan;

[0039] Figure 4 This is the image grayscale processing diagram;

[0040] Figure 5 Add Gaussian noise processing map to the image;

[0041] Figure 6 This is the image alternating Kalman column and row filtering processing diagram;

[0042] Figure 7 This is the processing diagram of the Log edge detection operator. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Example

[0045] like Figure 1-7 As shown, an AFM scanning image processing method based on alternating Kalman filtering and edge operator is proposed in one embodiment of the present invention, and includes the following steps.

[0046] S1. Grayscale the image obtained by AFM scanning to reduce the amount of data and facilitate subsequent processing operations;

[0047] A 512x512 pixel cell image is selected to simulate the image obtained by AFM scanning cells. If the image is a color image, the image is grayed. The grayed image has only one channel, which can reduce the computational burden and improve processing efficiency. In addition, graying can retain the main information of the image without losing important features of the image, which is helpful for the subsequent filtering process. The formula is used to gray the compressed image. The graying process requires converting the RGB565 format to RGB888 format before graying. The conversion formula is:

[0048]

[0049] After grayscale processing, the amount of image data is reduced by two-thirds. In order to further improve the real-time performance of the system, the image needs to be further compressed. In order to simulate the noise that may be encountered when AFM scans images in actual operation, Gaussian noise is added after grayscale processing.

[0050] N(x,y)~N(μ,σ 2 )

[0051] Where μ is the mean of Gaussian noise, usually set to 0, indicating no bias, and σ 2 is the variance of Gaussian noise, which determines the intensity of the noise.

[0052] I noisy (x,y)=I gray (x,y)+N(x,y)

[0053] I noisy (x, y) represents the pixel value of the image at position (x, y) after adding Gaussian noise. This is the new image we get, which contains the original image information and noise. gray (x, y) represents the pixel value of the grayscale image, and N(x, y) is the value of Gaussian noise at position (x, y), which is usually represented as a random variable and follows a normal distribution. The specific value of the noise is obtained by generating a random number with a specific mean and standard deviation.

[0054] S2. The grayscale image is compressed using an image compression algorithm based on a generative adversarial network to reduce the data dimension and further reduce the amount of data that needs to be processed by the alternating Kalman column-row filtering;

[0055] Image compression is performed on the grayscale image using an image compression algorithm based on a generative adversarial network. This step mainly performs lossless compression on the image. After the image is processed, the data dimension is reduced to generate a more compact image. The amount of input data for the subsequent alternating Kalman column-row filtering will be significantly reduced, and the matrix operations required for the Kalman column-row filtering will be calculated in a smaller feature space based on GANs, which can reduce the computational complexity of the filter and improve efficiency.

[0056] First, train GAN, and the loss function is:

[0057] L(G,D)=E x～pdata [logD(x)]+E z～pz [log(1-D(G(z))]

[0058] Where G is the generator, D is the discriminator, and p data is the true image distribution, p z is the potential space distribution, x is the real input image, which comes from p data , z is distributed from the latent space p z The sampled noise vector, L, is the loss between the generator and the discriminator, with the goal of minimizing the probability that a fake image generated by the generator is correctly identified as fake by the discriminator during training. After training, the generator is used to compress the AFM image.

[0059] I compressed =G(I input )

[0060] Among them, I input is the input image, I compressedTo compress the output image, this step compresses the image to help reduce computational and storage requirements while maintaining image quality, making subsequent processing more efficient.

[0061] S3, performing Kalman column-row filtering on the grayscale image, improving the image quality through iteration, so as to achieve high-precision AFM image processing;

[0062] The image after graying and adding Gaussian noise is processed by alternating Kalman column-row filtering. First, the data of the grayed image is acquired, and the image features are modeled as states for subsequent prediction and updating. Then, the previous features are predicted, the current state is predicted, and the possible dynamic changes are considered. The prediction covariance calculates its uncertainty, and the prediction result is corrected with the new observation value to improve the accuracy. Then, the Kalman gain is calculated to decide how to balance the importance of prediction and observation data, correct the current state estimate, update the covariance, adjust the assessment of state uncertainty, reflect the information given by the newly acquired data, and finally iterate. Through continuous iteration, the quality of image restoration is gradually improved, and finally high-precision AFM image processing is achieved. The above is the whole process of alternating Kalman column-row filtering.

[0063] The specific process of alternating Kalman filtering columns and rows is as follows:

[0064] Before filtering, we first need to initialize the state and covariance matrix. Let x k is the state vector at the current moment, usually the pixel value of the image. The image can be represented as a matrix I.

[0065] Initial state estimation: select an initial state in the image (such as the first pixel column)

[0066]

[0067] This is the first column of pixel values; initialize the covariance and set the initial covariance matrix P 0|0 .

[0068] Further define the model: set the state space model, define the state transition model and observation model;

[0069] State transition model:

[0070] x k =Fx k-1 +w k

[0071] Among them, w k is the process noise, is zero-mean Gaussian noise, and F is the state transfer matrix, which is usually set to the identity matrix:

[0072] F=I

[0073] Observation model:

[0074] z k =Hx k +v k

[0075] Among them, v k H is the observation matrix, which is usually the identity matrix:

[0076] H=I

[0077] Process each column j by column:

[0078] First, make a time prediction:

[0079] Predict current status:

[0080]

[0081] Update the covariance matrix:

[0082] P j,k|k-1 =FP j,k-1|k-1 F T +Q

[0083] Next, perform observation updates:

[0084] Calculate the Kalman gain:

[0085] K j,k =P j,k|k-1 H T (HP j,k|k-1 H T +R) -1

[0086] Update status:

[0087]

[0088] Update the covariance matrix:

[0089] P j,k|k =(IK j,k H)P j,k|k-1

[0090] After processing each column, proceed row by row, with similar steps:

[0091] Processing of each column i:

[0092] First, make a time prediction:

[0093] Predict current status:

[0094]

[0095] Update the covariance matrix:

[0096] P i,k|k-1 =FP i,k-1|k-1 F T +Q

[0097] Next, perform observation updates:

[0098] Calculate the Kalman gain:

[0099] K i,k =P i,k|k-1 H T (HP i,k|k-1 H T +R) -1

[0100] Update status:

[0101]

[0102] Update the covariance matrix:

[0103] P i,k|k =(IK i,k H)P i,k|k-1

[0104] is the initial state estimate, which represents the predicted value of the state at the initial moment, and the state vector x k Indicates the current system state. In AFM image processing, x k It is usually the value of the image pixel (or pixel row or column) at the current moment, which carries the key information about the image. The initial covariance P 0|0 Represents the uncertainty of the initial state estimate, usually set to a Gaussian noise covariance matrix. It describes our confidence in the initial state. The state transition matrix F describes how the system state is transferred from time k-1 to time k. For AFM images, it is usually set to the unit matrix I to represent the state (pixel value) over time. The process noise w k is the innovation term affected by random noise, usually assumed to be zero-mean Gaussian noise, representing the uncertainty of the model, and its covariance is Q. The observation matrix H is the matrix that maps the state space to the observation space. In most cases, the unit matrix I is also selected for image processing, indicating that we directly observe the state and observe the noise v k The random noise introduced during the observation process is usually assumed to be zero-mean Gaussian noise, which reflects the measurement error. Its covariance is R, and the predicted state and The predicted value of the system state before the observation update. It is calculated based on the estimate of the state at the previous moment and the state transition model. The predicted covariance P j,k|k-1 With P i,k|k-1 After the observation is updated, the update of the uncertainty of the current state estimate is usually smaller than the predicted covariance, and the Kalman gain K j,k With K i,k The gain used to balance the weight between the predicted value and the observed value is equivalent to the weight given to the observed data when updating the state. A larger value means a higher degree of trust in the observation and a higher confidence in the updated state. and The current state estimate is obtained by combining the predicted value and the observed value, taking into account the influence of process noise and observation noise, and updating the covariance P j,k|k With P i,k|k After the observation is updated, the update of the uncertainty of the current state estimate is usually smaller than the prediction covariance.

[0105] Finally, the above column and row filtering processes are combined, and each row and column are filtered in the order of column first and row later until the entire image is processed. Finally, the image processed by the alternating Kalman column and row filtering will be x k|k The output is in the form of , which can effectively remove noise and retain the important features of the image.

[0106] S4, using the log edge detection operator to extract edge features from the image obtained after the alternating Kalman filter processing, simplifying the complexity in the image and extracting useful structural information;

[0107] Use the Log edge detection operator to extract edge features from the image. The formula is as follows:

[0108]

[0109] The Log operator enhances the high-frequency features of the image (such as edges), and σ controls the width of the Gaussian filter.

[0110] The image after Kalman filtering Perform a convolution operation:

[0111]

[0112] Through convolution, edge features are extracted and the image contour is enhanced. After the alternating Kalman column-row filter, the application of the Log operator can further refine the details of the image and improve the image quality. The alternating Kalman column-row filter provides a dynamic noise suppression strategy, and the edge detection of the Log operator supplements this deficiency, making the final image not only clear but also rich in information.

[0113] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An AFM scanning image processing method based on alternating Kalman filtering and edge operator, characterized in that: The following steps are involved: S1, grayscale the image obtained by AFM scanning, which reduces the amount of data and facilitates the processing operations of S3 and S4; S2. The grayscale image is compressed using an image compression algorithm based on a generative adversarial network to reduce the data dimension and the amount of data that needs to be processed by the alternating Kalman column-row filtering; S3, performing Kalman column-row filtering on the grayscale image, improving the image quality through iteration, so as to achieve high-precision AFM image processing; S4. Use the log edge detection operator to extract edge features from the image obtained after the alternating Kalman filter processing to simplify the complexity in the image.

2. The AFM scanning image processing method based on alternating Kalman filtering and edge operator according to claim 1, characterized in that: In S1, the target image is grayed using the formula: I gray (x,y)=0.2989·R(x,y)+0.5870·G(x,y)+0.1140·B(x,y) Among them, R, G, B are the red, green and blue channel values ​​of the color image at the corresponding position, I gray (x, y) is the pixel value of the grayscale image at position (x, y). Grayscale processing can simplify the image data so that the alternating Kalman column-row filter and the edge operator only need to process one channel when processing the image, reducing the computational complexity and improving the real-time performance of the system. The image needs to be further compressed.

3. The AFM scanning image processing method based on alternating Kalman filtering and edge operator according to claim 1, characterized in that: In S2, the grayscale processed image is compressed using an image compression algorithm based on a generative adversarial network: First, train GAN, and the loss function is: Where G is the generator, D is the discriminator, and p data is the true image distribution, p z is the potential space distribution, x is the real input image, which comes from p data , z is distributed from the latent space p z The sampled noise vector, the loss between the L generator and the discriminator, aims to minimize the probability that the fake image generated by the generator is correctly identified as fake by the discriminator during training. After the training is completed, the generator is used to compress the AFM image.

4. The AFM scanning image processing method based on alternating Kalman filtering and edge operator according to claim 1, characterized in that: In S3, before performing the alternating Kalman column-row filtering, data is first acquired from the grayscale processed image, and the image features are modeled as states to facilitate subsequent prediction and updating. Prediction and updating are the two main steps of the alternating Kalman column-row filtering, and the formula used is as follows: Predict current status: Update the covariance matrix: P s,k|k-1 =FP s,k-1|k-1 F T +Q Calculate the Kalman gain: K s,k =P s,k|k-1 H T (HP s,k|k-1 H T +R) -1 To predict or update the state, P s,k|k-1 To predict or update the covariance, the predicted value of the uncertainty of the current state, K s,k Kalman gain is the gain used to balance the weight between the predicted value and the observed value, which is equivalent to the weight given to the observed data when updating the state; a larger value means a greater degree of trust in the observation. Kalman gain determines how to balance the importance of predicted and observed data and is an important indicator for updating. Alternating Kalman column-row filtering is based on the continuous iteration of prediction and update steps to gradually improve the quality of image restoration and ultimately achieve high-precision AFM image processing.

5. The AFM scanning image processing method based on alternating Kalman filtering and edge operator according to claim 1, characterized in that: In S4, Log uses an edge detection operator to extract edge features from an image, and the formula is as follows: x and y are the relative positions of the current pixel in the image coordinate system. x represents the horizontal displacement and y represents the vertical displacement. In the convolution operation, these two parameters help determine the position of the filter application. σ is the standard deviation of Gaussian smoothing, which controls the width of the Gaussian function. It affects the shape and size of the filter. A large σ value will result in a wider smoothing effect, which helps to remove high-frequency noise, while a smaller σ value can capture more subtle edge features. This term is a normalization factor, which is used to ensure that the total energy of the operator remains consistent during integration, so that the output of the operator is not affected by the size of σ, which makes the edge detection result more stable. This part is a directional information, which means that the farther away from the center point, the smaller the impact; it forms positive and negative features near the edge. By adjusting this value to enhance edge contrast, it helps to distinguish the positive and negative changes in the edge position and enhance the edge detection effect; It is a component of the Gaussian function, which makes the weight of pixels farther away from the center point lower, and the shape presents circular symmetry, which conforms to the Gaussian distribution characteristics. Through this item, the Log operator achieves a good local smoothing effect, while producing a strong response at the edge position, effectively capturing edge information.

Images