Atomic force microscope cell image super-resolution reconstruction method

Abstract

The invention belongs to the technical field of image processing and biology, and discloses an atomic force microscope cell image super-resolution reconstruction technology. Comprising the following steps: S1, acquiring an AFM cell image, and establishing an AFM cell data set; s2, constructing a super-resolution network model; s3, training a network model; and S4, carrying out super-resolution reconstruction. According to the method, the super-resolution network of the AFM cell image is established, and the super-resolution reconstruction of the AFM cell image is completed through the methods of feature extraction, a self-adaptive edge extraction module, addition of a self-adaptive space fusion structure, network structure optimization through joint back projection and the like in the network. The problems that texture of the reconstructed image disappears and artifacts are generated are solved.

Description

Technical Field

[0001] The present invention relates to image processing technology and the biological field, and specifically to a method for super-resolution reconstruction of atomic force microscope cell images. Background Art

[0002] As a high-resolution imaging technology, atomic force microscopy plays an important role in cell biology research. It can help researchers observe the morphology and stress characteristics of the cell surface at the nanoscale, providing key data for revealing cell functions such as cell-cell interactions and cell responses to the microenvironment. However, AFM cell images usually have the problem of low resolution, and the noise and complex cell structure details contained in the images are difficult to present clearly, severely limiting the in-depth development of cell biology research.

[0003] Currently, although deep learning technology has made significant progress in the field of image super-resolution, models such as SRGAN have shown certain advantages in general image super-resolution reconstruction. However, when faced with AFM cell images, these methods still have many deficiencies. AFM cell images have unique spectral information, and different frequency components have quite different effects on image quality. Traditional models are difficult to effectively process this characteristic, easily leading to problems such as the loss of important details and inaccurate reconstruction of cell structures, and cannot meet the strict requirements for image quality in cell biology research. Ferdousi et al. proposed a method combining super-resolution convolutional neural network (SRCNN) with AFM. This achievement is of great significance in the field of general image super-resolution and also points out a new direction for the research on super-resolution of AFM cell images. Ferdousi et al. developed SRGAN and its variants based on GAN, aiming to generate more realistic and detail-rich high-resolution cell images, and significantly improving the perceptual quality of the reconstructed images through an adversarial training mechanism.

[0004] However, these models still have problems in restoring the details of cell images. The texture disappears after cell image reconstruction, and artifacts appear in the reconstructed images. Therefore, we propose a method for super-resolution enhanced reconstruction of atomic force microscope cell images to solve this problem. Summary of the Invention

[0005] (1) Technical Problems to be Solved

[0006] In view of the deficiencies of the prior art, the present invention provides a method for super-resolution reconstruction of atomic force microscope cell images, solving the problems raised in the above background art.

[0007] (2) Technical Solutions

[0008] The present invention specifically adopts the following technical solutions to achieve the above objectives:

[0009] A method for super-resolution reconstruction of atomic force microscope cell images, comprising the following steps:

[0010] S1. Build an AFM cell dataset;

[0011] S11. Culture cells, select appropriate cells in the laboratory with a growth cycle of one to two days. Select cells with good cell activity for subculture to ensure the source of the cell dataset;

[0012] S12. Collect cell images on the AFM to obtain high-resolution cell images. Perform data augmentation on the collected cell images to expand the number of high-resolution cell images, and then perform downsampling on the high-resolution cell images to obtain corresponding low-resolution cell images;

[0013] S13. Use the low-resolution and high-resolution images as a pair of training samples to obtain an AFM cell dataset;

[0014] S2. Build a super-resolution network model;

[0015] The super-resolution network model includes a coarse super-resolution stage and a back-projection and refinement stage. The two stages are the same and both contain a feature extraction module, an edge extraction module, a pre-trained StyleGAN module, and corresponding encoders and decoders. The overall structure designs an edge fusion and a spatial fusion structure and adopts a back-projection connection method;

[0016] S3. Train the super-resolution network model;

[0017] Sort out the AFM cell dataset, divide the high and low AFM cell image pairs into a training set and a validation set according to a ratio of 8:2; during training, randomly select images from the training set and input them into the model, and adjust the training parameters of the training model according to the difference between the prediction result and the true label through backpropagation to minimize the loss function; after each training, evaluate the model with the validation set; until the loss function converges to obtain a trained model;

[0018] S4. Perform super-resolution reconstruction on the AFM cell images;

[0019] Input the selected cell AFM images into the trained model to obtain high-resolution AFM cell images after super-resolution reconstruction;

[0020] Furthermore, in step S2, the feature extraction module is at the front end of the network, extracts features of different scales through a group of parallel multi-scale convolutional layers, and the adaptive edge module is located in the layer after the feature extraction module. Using a dynamic fusion strategy, it adaptively fuses edge information with other image features according to the local features of the cells;

[0021] Further, in step S2, the spatial fusion structure refers to asymmetrically fusing the encoder features and StyleGAN features to improve the reconstruction effect;

[0022] Further, in step S2, the joint back-projection mechanism refers to dynamically adjusting the projection weights according to the local features and error distribution of the image, and introducing structural perception constraints to improve the biological rationality of the reconstructed image;

[0023] Further, in step S3, the mean squared error (MSE) is used to construct the perceptual loss as the loss function, and the mathematical expression of the loss function is:

[0024]

[0025] In the above formula, n represents the number of samples, y i represents the true value, represents the predicted value;

[0026] (III) Beneficial effects

[0027] Compared with the prior art, the present invention provides a method for super-resolution reconstruction of atomic force microscope cell images, having the following beneficial effects:

[0028] 1. The present invention improves the resolution of AFM cell images, accurately restores the microscopic structure and detailed information of cell images, provides strong support for cell biology research, and helps to deeply explore cell functions and mechanisms.

[0029] 2. The present invention realizes the super-resolution reconstruction of AFM cell images by building a super-resolution network model, reduces the time for obtaining high-resolution AFM cell images, reduces the difficulty of obtaining biological cell information, and reduces the experimental loss during the scanning of living cell biological experiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flowchart of the method of the present invention;

[0031] Figure 2 is a schematic diagram of the super-resolution network structure;

[0032] Figure 3 is a schematic diagram of the structure of the feature extraction module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0034] Embodiment

[0035] As shown in the figure, a method for super-resolution reconstruction of atomic force microscope cell images proposed by an example of the present invention includes the following steps:

[0036] S1. Build an AFM cell dataset;

[0037] S11. Culture cells, select liver cancer cells in the laboratory, with a growth cycle of one day. Select cells with good cell activity for passage to ensure the source of the cell dataset;

[0038] S12. Collect cell images on the AFM to obtain high-resolution cell images, perform data augmentation on the collected cell images, expand the number of high-resolution cell images, and then perform downsampling on the high-resolution cell images to obtain corresponding low-resolution cell images;

[0039] S13. Use the low-resolution and high-resolution images as a pair of training samples to obtain an AFM cell dataset;

[0040] S2. Build a super-resolution network model;

[0041] The super-resolution network model includes a coarse super-resolution stage and a back-projection and refinement stage. The two stages are the same and both include a feature extraction module, an edge extraction module, a pre-trained StyleGAN module, and corresponding encoders and decoders. The overall structure designs an edge fusion and a spatial fusion structure and adopts a back-projection connection method;

[0042] Furthermore, the feature extraction module is at the front end of the network, extracting features of different scales through a group of parallel multi-scale convolutional layers. The adaptive edge module is located in the layer behind the feature extraction module, using a dynamic fusion strategy to adaptively fuse edge information with other image features according to the local features of the cells;

[0043] Furthermore, the spatial fusion structure refers to asymmetrically fusing the encoder features and the StyleGAN features to improve the reconstruction effect;

[0044] Furthermore, the joint back-projection mechanism refers to dynamically adjusting the projection weights according to the local features and error distribution of the image, introducing a structure-aware constraint to improve the biological rationality of the reconstructed image;

[0045] S3. Train the super-resolution network model;

[0046] Organize the AFM cell dataset, and divide the high- and low-AFM cell image pairs into a training set and a validation set according to a ratio of 8:2. During training, randomly select images from the training set and input them into the model. Adjust the training parameters of the training model through backpropagation according to the difference between the prediction result and the true label to minimize the loss function. After each training, evaluate the model with the validation set until the loss function converges to obtain a trained model.

[0047] Further, use the mean square error (MSE) to form the perceptual loss as the loss function. The mathematical expression of the loss function is:

[0048]

[0049] In the above formula, n represents the number of samples, y i represents the true value, represents the predicted value;

[0050] S4. Perform super-resolution reconstruction on the AFM cell images;

[0051] Input the selected cell AFM images into the trained model to obtain high-resolution AFM cell images after super-resolution reconstruction.

[0052] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A super-resolution reconstruction method for atomic force microscopy cell images, characterized in that: The following steps are involved: S1. Self-built AFM cell dataset; S11. Cultivate cells and select appropriate cells in the laboratory. The growth cycle is one to two days. Select cells with good cell activity for passaging to ensure the source of cell data sets; S12. collecting cell images on the AFM to obtain high-resolution cell images, performing data enhancement on the collected cell images to increase the number of high-resolution cell images, and then downsampling the high-resolution cell images to obtain corresponding low-resolution cell images; S13. Use low-resolution and high-resolution images as a pair of training samples to obtain an AFM cell dataset; S2. Build a super-resolution network model; The super-resolution network model includes a coarse super-resolution stage and a back-projection and refinement stage. The two stages are the same, both containing a feature extraction module, an edge extraction module, a pre-trained StyleGAN module, and corresponding encoders and decoders. The overall structure designs edge fusion and spatial fusion structures, and adopts a back-projection connection method; S3. Train super-resolution network model; The AFM cell dataset was sorted, and the high and low AFM cell image pairs were divided into training set and validation set in a ratio of 8:

2. During training, images were randomly selected from the training set and input into the model. The training parameters of the training model were adjusted through back propagation according to the difference between the predicted results and the true labels to minimize the loss function. After each training, the model was evaluated with the validation set until the loss function converged and a trained model was obtained. S4. Super-resolution reconstruction of AFM cell images; The selected cell AFM images are input into the trained model to obtain high-resolution AFM cell images after super-resolution reconstruction.

2. The method for super-resolution reconstruction of cell images using an atomic force microscope according to claim 1, characterized in that: In step S12, the image enhancement algorithm refers to using a geometric transformation algorithm to rotate the AFM image at different angles, such as thirty degrees and sixty degrees clockwise, to simulate different orientations and distributions of cells in the sample by changing the direction and position of the cell image.

3. The method for super-resolution reconstruction of cell images using an atomic force microscope according to claim 1, characterized in that: In step S2, the feature extraction module is at the front end of the network, and extracts features of different scales through a set of parallel multi-scale convolutional layers. The adaptive edge module is located in the layer after the feature extraction module, and uses a dynamic fusion strategy to adaptively fuse edge information with other image features based on local cell features.

4. The method for super-resolution reconstruction of cell images using an atomic force microscope according to claim 1, characterized in that: In step S2, the spatial fusion structure refers to asymmetric fusion of encoder features and StyleGAN features to improve the reconstruction effect.

5. The method for super-resolution reconstruction of cell images using an atomic force microscope according to claim 1, characterized in that: In step S2, the joint back-projection mechanism refers to dynamically adjusting the projection weights according to the local features of the image and the error distribution, and introducing structure-aware constraints to improve the biological plausibility of the reconstructed image.

6. The method for super-resolution reconstruction of cell images using an atomic force microscope according to claim 1, characterized in that: In step S3, the mean square error (MSE) is used to form the perceptual loss as the loss function, and the mathematical expression of the loss function is: In the above formula, n represents the number of samples, y i represents the true value, Represents the predicted value.

7. The method for super-resolution reconstruction of cell images using an atomic force microscope according to claim 1, characterized in that: Multi-module collaboration improves model performance, and the adaptive strategy is cleverly used to dynamically adjust model parameters to increase the applicability of the designed network.

Images