A cell image reconstruction method, device, computer equipment and storage medium
By constructing a composite convolution U network MCU-Net, asymmetric expansion convolution and multi-layer residual local feature modules are used to solve the problem of low cell image reconstruction efficiency under low light conditions, and efficient feature extraction and fusion are achieved. It improves the resolution and detail resilience of cell images, and is suitable for real-time observation of cell dynamic changes.
Patent Information
- Application Number
- CN202411321349.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-23
AI Technical Summary
The prior art has low efficiency in cell image reconstruction under low light conditions, resulting in loss of detailed information, high computational burden, slow processing speed, and unsuitable for real-time observation of cell dynamic changes.
The composite convolution U network MCU-Net is constructed, using asymmetric expansion convolution and multi-layer residual local feature modules, and introducing a hollow space convolution pooling pyramid, and a cell structured light microscopy reconstruction model is generated by training structured light image samples to realize Efficient feature extraction and fusion.
It significantly improves the resolution and detail resilience of cell images, reduces the computational complexity, maintains the naturalness and continuity of the image, improves the reconstruction speed and efficiency, and is suitable for real-time observation of cell dynamic changes.
Smart Images

Figure CN118864254B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image processing, and in particular relates to a cell image reconstruction method, device, computer equipment and storage medium. Background Art
[0002] In cell research, super-resolution imaging technology is used to reveal cell behavior non-destructively, and structured illumination microscopy, in particular, has shown its unique advantages in this field. Super-resolution technology based on structured light uses specific optical patterns to excite samples and reconstructs high-resolution images by integrating the frequency domain information of multiple images captured at different angles. This method significantly improves the spatial resolution of optical imaging systems. However, this technology usually requires high-intensity illumination and multiple image acquisitions to generate a single high-resolution image. This process not only significantly reduces the imaging speed, but also increases the time that the sample is exposed to strong light, leading to increased phototoxicity and photobleaching problems.
[0003] In the existing technology, low light is used to reduce sample exposure problems. However, under low light conditions, the signal-to-noise ratio of the image will be significantly reduced, resulting in loss of detail information. Therefore, in order to obtain more image details, the existing technology usually involves millions of weight parameters for image reconstruction, resulting in a huge computational burden and slow processing speed. It is not suitable for real-time observation of dynamic changes in cells, and also increases the energy consumption of the equipment and the complexity of operation. Summary of the invention
[0004] In order to solve the problem of low image reconstruction efficiency under low light conditions, the present invention provides a cell image reconstruction method, device, computer equipment and storage medium.
[0005] In order to achieve the above object, the present invention provides the following technical solutions:
[0006] A cell image reconstruction method, the method comprising:
[0007] A composite convolutional U network MCU-Net is constructed; the MCU-Net is based on the architecture of the U-net network, including an input module, an encoder, a decoder and an image reconstruction module; the dilated convolution is introduced into the asymmetric convolution to obtain the asymmetric dilated convolution AD, and the AD is used as the input module; multiple residual local feature modules RLFB are used as multi-layer encoders; the RLFB is connected with multi-layer residuals, and the atrous spatial convolution pooling pyramid ASPP is introduced to obtain the enhanced feature aggregation module EFAB, and the multiple EFABs corresponding to the RLFB are used as multi-layer decoders; the MCU-Net is trained by using the structured light image samples containing cells to obtain a cell structured light microscopy reconstruction model;
[0008] A target cell image is acquired, and the target cell image is input into a cell structure light microscopy reconstruction model. The input target cell image is preliminarily identified by the input module to obtain an initial feature map. Local features of the initial feature map are extracted by a multi-layer encoder to obtain an enhanced feature map. The enhanced feature map is feature-fused by a multi-layer decoder to obtain a comprehensive feature map. Finally, the comprehensive feature map is reconstructed and output by an image reconstruction module to obtain a reconstructed target image.
[0009] Optionally, introducing the dilated convolution into the asymmetric convolution to obtain the asymmetric dilated convolution AD includes:
[0010] Add dilation factor between kernel elements of asymmetric convolution , get the vertical convolution for:
[0011] ;
[0012] Horizontal Convolution for:
[0013] ;
[0014] Square Convolution for:
[0015] ;
[0016] The vertical convolution, horizontal convolution and square convolution are subjected to feature fusion to obtain the asymmetric dilated convolution AD:
[0017] ;
[0018] Among them, the convolution kernel size is , Represents the number of units that the convolution kernel extends outward from the center point, Represents the input feature map In the center The position offset is and The pixel feature value of , and They represent convolution kernels that extend vertically, horizontally, and square convolution kernels, respectively. represents the location of the convolution kernel weight, b Represents a bias term.
[0019] Optionally, the extracting local features of the initial feature map using the multiple residual local feature modules RLFB as a multi-layer encoder to obtain an enhanced feature map includes:
[0020] The initial feature map The convolution kernel is The convolution operation is then applied to Re LU Activation function gets feature map ;
[0021] And the feature map after multiple RLFB input Perform residual connection and get output ;
[0022] Through the spatial attention module ESA Perform feature transformation And apply the activation function , generate attention weights, and then compare the attention weights with Multiply element by element to get the enhanced feature map .
[0023] Optionally, performing multi-layer residual connection on the RLFB includes:
[0024] The first The RLFB module performs the previous layer feature map Perform feature extraction and obtain the output feature map ;
[0025] Will With the previous layer feature map And the feature maps of all previous layers Perform residual connection to generate updated feature map .
[0026] Optionally, the introducing the atrous spatial convolutional pooling pyramid ASPP includes:
[0027] Each convolution output of the RLFB after the residual connection is concatenated with the output of the previous layer in the channel dimension and further fused through 1x1 convolution.
[0028] Optionally, the structured light image samples include a low-resolution structured light image and its corresponding super-resolution structured light image; and the training of MCU-Net using the structured light image samples containing cells to obtain the cell structured light microscopic reconstruction model includes:
[0029] The low-resolution structured light image is input into MCU-Net to obtain a predicted high-resolution image. The MCU-Net is trained with the goal of minimizing the loss function between the predicted high-resolution image and the corresponding super-resolution structured light image to obtain a cell structured light microscopy reconstruction model.
[0030] Optionally, by taking the mean square error MSE and structural similarity index SSIM Combined, the loss function is constructed:
[0031] ;
[0032] in, is the ratio coefficient of the structural correlation index, used to balance SSIM and MSE The relative contribution of S T and S P represent the true value and the model predicted value respectively.
[0033] A cell image reconstruction device, comprising:
[0034] A construction module is used to construct a composite convolutional U network MCU-Net; the MCU-Net is based on the architecture of the U-net network, including an input module, an encoder, a decoder and an image reconstruction module; the dilated convolution is introduced into the asymmetric convolution to obtain an asymmetric dilated convolution AD, and the AD is used as the input module; multiple residual local feature modules RLFB are used as multi-layer encoders; the RLFB is connected with multiple layers of residuals, and the hole space convolution pooling pyramid ASPP is introduced to obtain an enhanced feature aggregation module EFAB, and multiple EFABs corresponding to the RLFB are used as multi-layer decoders;
[0035] A training module is used to train MCU-Net using structured light image samples containing cells to obtain a cell structured light microscopy reconstruction model;
[0036] A reconstruction module is used to obtain a target cell image, input the target cell image into a cell structure light microscopy reconstruction model, perform preliminary recognition on the input target cell image through the input module to obtain an initial feature map, extract local features of the initial feature map through a multi-layer encoder to obtain an enhanced feature map, perform feature fusion on the enhanced feature map through a multi-layer decoder to obtain a comprehensive feature map, and finally reconstruct and output the comprehensive feature map through an image reconstruction module to obtain a reconstructed target image.
[0037] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the cell image reconstruction method is implemented.
[0038] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned cell image reconstruction method when executing the program.
[0039] The cell image reconstruction method provided by the present invention has the following beneficial effects:
[0040] By constructing a composite convolutional U network and specifically constructing an asymmetric dilated convolution to achieve multi-scale feature extraction, the effects of scale scaling and primary feature extraction are enhanced to avoid detail loss in cell images. By constructing an enhanced feature aggregation module to achieve residual connection design, more efficient feature extraction and fusion are achieved. While significantly improving the resolution and detail restoration capabilities of microbial cell images, the computational complexity of related details is further reduced. Not only can the naturalness and continuity of the image be maintained when processing areas with rich details, but the image can also be processed more quickly while maintaining high reconstruction quality, thereby improving the overall reconstruction speed and efficiency, which is conducive to real-time observation of dynamic changes in cells. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0042] Figure 1 The figure is a schematic flow chart of a cell image reconstruction method provided by the present invention according to an exemplary embodiment.
[0043] Figure 2 A structural block diagram of a composite convolutional U network provided according to an exemplary embodiment of the present invention.
[0044] Figure 3 The present invention is a block diagram of a cell image reconstruction device according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION
[0045] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.
[0046] This paper uses sample data obtained using structured light illumination under low light conditions to develop a lightweight algorithm Multi-Convolutional U-Net (MCU-Net) with super-resolution capabilities. A composite convolutional structure is used to replace the traditional convolution operation, and an attention mechanism is introduced to enhance its ability to process microscopic image super-resolution. These innovative designs allow MCU-Net to demonstrate excellent performance in processing microscopic images, and its superiority and effectiveness in super-resolution tasks are verified through extensive experiments on multiple benchmark datasets.
[0047] The technical solutions provided by various embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.
[0048] First, the present invention provides a cell image reconstruction method, specifically, Figure 1 As shown, the following steps are included:
[0049] S101. Build a composite convolutional U network MCU-Net.
[0050] The MCU-Net is based on the U-Net architecture, including an input module, an encoder, a decoder, and an image reconstruction module. Figure 2 shown.
[0051] In one embodiment, dilated convolution is introduced into asymmetric convolution to obtain asymmetric dilated convolution AD, and AD is used as an input module. The input image is preliminarily recognized through the input module to obtain an initial feature map.
[0052] For example, when performing super-resolution reconstruction of living cells in a low-light environment, it becomes more critical to accurately capture highly detailed changes and perform effective feature extraction. In order to more effectively capture the details and texture information of the image, especially to show higher sensitivity and accuracy in processing lines and edges, the traditional 3×3 standard convolution is discarded as the first layer in the network to extract shallow features from the input image, because when processing complex low-light microscopic images, the efficiency and flexibility of the standard convolution kernel in capturing features of different directions and scales are limited. Based on the asymmetric convolution block (ACB), the present invention proposes a more effective asymmetric dilated convolution (AD) to further enhance the effects of scale scaling and primary feature extraction.
[0053] The asymmetric dilated convolution AD retains the ability of ACB to capture detailed edges, performs fine edge and detail processing on the input feature map, and uses asymmetric convolution operations to enhance the expressiveness of local features. Specifically, AD uses 1×3, 3×3, and 3×1 convolution kernels in the horizontal and vertical directions, respectively. This separate processing strategy can more accurately capture and enhance the line and edge information in the image, thereby providing a more detailed local feature expression for subsequent feature extraction and analysis. At the same time, in order to capture a wider range of contextual information, dilated convolution (DC) is introduced to expand the receptive field. By increasing the space between the convolution kernel elements (i.e., the dilation factor ), which expands the receptive field and allows the model to cover a larger input area without adding additional computational burden.
[0054] For example, by using 1×3, 3×3, and 3×1 convolution kernels with a dilation rate of 2, the spatial coverage of the model is significantly enhanced, which is particularly important when processing complex microscopic images, allowing the model to capture and interpret microstructures and their extensive contextual information while maintaining high resolution. This process involves the extended application of matrix calculations, assuming that the dilation rate is , adding dilation factors between the convolution kernel elements of asymmetric convolution , the convolution kernel size is , then for each Position, we can get vertical convolution for:
[0055] ;
[0056] Horizontal Convolution for:
[0057] ;
[0058] Square Convolution for:
[0059] ;
[0060] The vertical convolution, horizontal convolution and square convolution are subjected to feature fusion to obtain the asymmetric dilated convolution AD:
[0061] ;
[0062] in, It is a square convolution, using a traditional square convolution kernel Perform a standard two-dimensional convolution operation, usually covering a area, and Represents the input feature map In the center The position offset is and Pixel feature values, such square convolution is helpful to capture the basic features of the image, Represents the number of units that the convolution kernel extends outward from the center point, represents the location of the convolution kernel weight, b Represents a bias term, which is used to make an overall adjustment to the final output of the convolution, making the output of the convolution layer more flexible. And horizontal convolution Use convolution kernels that extend specifically in the vertical and horizontal directions, respectively. and , the specific size is and , these operations focus on strengthening the lines and edge information of the image. The vertical convolution kernel captures vertical features, while the horizontal convolution kernel emphasizes horizontal details, allowing the model to understand the structure and directional characteristics of the image in more detail. Finally, the convolution outputs of these different directions are added to form the comprehensive output of AD This step combines features from all directions to provide a fully enhanced feature map. This fusion not only improves the model's ability to recognize low-contrast features, but also enhances its understanding of deep features of complex scenes, thereby significantly improving imaging quality and accuracy.
[0063] In another embodiment, a plurality of residual local feature blocks (RLFBs) are used as multi-layer encoders to extract local features of an initial feature map to obtain an enhanced feature map.
[0064] Specifically, RLFB, as a deep learning component focused on enhancing local feature processing of images, is designed by integrating Conv-ReLU stacking and Enhanced Spatial Attention (ESA) modules, which significantly enhances the perception of local spatial details of images.
[0065] For example, RLFB extracts local features by stacking multiple layers of convolution and ReLU activation functions, and uses the enhanced spatial attention mechanism of the ESA module to focus on important areas in the image, combined with multi-scale processing to fully capture details. Residual connections ensure efficient feature transfer and fusion, improving the expressiveness and robustness of the model. Each processing unit in RLFB independently evaluates the features of the input image region, performs deep feature extraction and refinement, and thus performs well in complex visual tasks such as detail restoration and texture enhancement. Through this design, RLFB can gradually restore the details in the image, ensure that key information is not lost at each processing stage, and maintain the overall accuracy and integrity of image reconstruction.
[0066] ;
[0067] ;
[0068] ;
[0069] In RLFB, the input feature map is represented as . The initial feature map The convolution kernel is The convolution operation is then applied to LU Activation function gets feature map . With the final input feature map Perform residual connection and get output . ESA module Perform feature transformation And apply the activation function , generate attention weights, and then compare the attention weights with Multiply element by element to get the enhanced output feature map Through convolutional layers and attention mechanisms, the model can independently extract, evaluate, and process features in each local area of the image at a fine-grained level, thereby performing well in complex visual tasks such as detail restoration and texture enhancement.
[0070] In another embodiment, the RLFB is subjected to multi-layer residual connection, and the atrous spatial convolution pooling pyramid ASPP is introduced to obtain an enhanced feature aggregation block (EFAB). Multiple EFABs corresponding to the RLFB are used as multi-layer decoders to perform feature fusion on the enhanced feature map to obtain a comprehensive feature map. Finally, the comprehensive feature map is reconstructed and outputted through an image reconstruction module.
[0071] Based on the residual local feature module, the present invention innovatively proposes an enhanced feature aggregation module to extract and learn local features in the image, and uses residual connections to retain the input feature information while maintaining the coherence and correlation of these features at different scales.
[0072] For example, EFAB uses an enhanced spatial attention mechanism to focus on the details and global structure of images at different scales, thereby improving the richness and diversity of the overall feature representation. The specific feature processing process of EFAB is as follows:
[0073] ;
[0074] ;
[0075] In the above comprehensive formula of RLFB module and residual connection, The RLFB module performs the previous layer feature map Perform feature extraction and obtain the output feature map ; then With the previous layer feature map And the feature maps of all previous layers By weight and Perform residual connection to generate updated feature map ;For the first RLFB module, the feature map Depend on And the weighted input feature map By weight This design enables each layer to retain and gradually enhance feature representation, ensuring that the model can fully capture and integrate key information in the image.
[0076] In addition, the EFAB module also adds an improved Atrous Spatial Pyramid Pooling (ASPP) module. This improvement enhances the model's ability to process features of different scales by introducing a step-by-step expanded atrous convolution and a dynamic feature fusion mechanism. This module not only uses atrous convolutions with different dilation rates, but also adds a feature fusion step after each convolution layer to capture richer contextual information. Specifically, each convolution output of the RLFB after the residual connection is concatenated with the output of the previous layer in the channel dimension and further fused through a 1x1 convolution. This hierarchical and recursive fusion strategy significantly retains the information of the input features and enhances the diversity and richness of the feature representation, enabling excellent performance when processing images with complex structures and details, improving overall performance.
[0077] ;
[0078] ;
[0079] ;
[0080] ;
[0081] In the above formula of dilated convolution, Indicates that the expansion rate is of The convolution operation starts from the input feature map The extracted features, is through Convolution will dilate the convolution output And the input feature map The feature map obtained after splicing; in the comprehensive feature fusion formula, is the initial input feature map, Is to input F 0, F 3. F 6. F The output of 9 is concatenated along a specific dimension (usually the channel dimension), which represents the result obtained by concatenating multiple feature maps. The concatenation operation merges different feature maps along a specific dimension (usually the channel dimension), thereby retaining all the information in these feature maps. This means that the output of the network depends entirely on the merged results of all level features without additional feature processing or adjustment. The idea of residual learning is to promote information flow and gradient propagation by directly adding shallow information to the deep network, thereby helping to solve the gradient disappearance problem and improve the training efficiency of the model. After 1x1 convolution operation Perform residual connection, 1x1 convolution as a tool to compress and integrate features, can adjust and optimize F cat The multi-scale information in makes the output features more compact and efficient.
[0082] MCU-Net adopts a multi-module collaborative design, which significantly improves the resolution and detail restoration capabilities of microbial cell images by using residual connections, compound convolutions, multi-scale feature fusion, and spatial attention mechanisms.
[0083] S102. Train MCU-Net using structured light image samples containing cells to obtain a cell structured light microscopy reconstruction model.
[0084] In this step, firstly, a structured light image sample containing cells is acquired, and the structured light image sample includes a low-resolution structured light image and its corresponding super-resolution structured light image.
[0085] The low-resolution structured light image is input into MCU-Net to obtain a predicted high-resolution image. The MCU-Net is trained with the goal of minimizing the loss function between the predicted high-resolution image and the corresponding super-resolution structured light image to obtain a cell structured light microscopy reconstruction model.
[0086] In which, by taking the mean square error MSE and structural similarity index SSIM Combined, the loss function is constructed:
[0087] ;
[0088] in, is the ratio coefficient of the structural correlation index, used to balance SSIM and MSE The relative contribution of S T and S P represent the true value and the model predicted value respectively.
[0089] ;
[0090] in, represents the true value, represents the model prediction value, and are the width and height of the image in pixels respectively.
[0091] ;
[0092] in, and Represent the predicted image and the real image respectively; and The images are and The mean value of reflects the brightness information; and The images are and The variance of reflects the contrast information. is an image and The covariance of reflects the structural information. and A constant introduced to avoid the denominator being zero, usually and ,in is the dynamic range of pixel values, and is a very small constant, usually and .
[0093] In this way, by combining the mean square error and the structural similarity index as the loss function, the reconstruction effect of the model can be more comprehensively evaluated, which can better balance the pixel accuracy and structural information preservation and improve the overall quality of image reconstruction.
[0094] In addition, the network is trained by back-propagation method to determine the optimal parameters. In order to ensure that the model parameters can be optimized efficiently, the adaptive moment estimation optimization algorithm (Adam) is used in the training process. By iteratively adjusting the learning rate and other hyperparameter settings, the model gradually learns and extracts the subtle structures and complex features in the microbial cell images, and finally achieves high-precision and high-quality image restoration in the super-resolution reconstruction task.
[0095] S103, acquiring a target cell image, and inputting the target cell image into a cell structured light microscopy reconstruction model to obtain a reconstructed target image.
[0096] Specifically, the target cell image is input into the cell structure light microscopy reconstruction model, the input target cell image is preliminarily identified by the input module to obtain an initial feature map, the local features of the initial feature map are extracted by a multi-layer encoder to obtain an enhanced feature map, the enhanced feature map is feature fused by a multi-layer decoder to obtain a comprehensive feature map, and finally the comprehensive feature map is reconstructed and output by an image reconstruction module to obtain a reconstructed target image.
[0097] By adopting the above method, a composite convolutional U network is constructed, and an asymmetric dilated convolution is specifically constructed to realize multi-scale feature extraction, which enhances the effects of scale scaling and primary feature extraction to avoid detail loss of cell images. The residual connection design is realized by constructing an enhanced feature aggregation module, which realizes more efficient feature extraction and fusion. While significantly improving the resolution and detail restoration ability of microbial cell images, the computational complexity of related details is further reduced. Not only can the naturalness and continuity of the image be maintained when processing areas with rich details, but also the image can be processed more quickly while maintaining high reconstruction quality, thereby improving the overall reconstruction speed and efficiency, which is conducive to real-time observation of dynamic changes of cells.
[0098] Secondly, the present invention also provides a cell image reconstruction device, such as Figure 3 As shown, including:
[0099] A construction module 301 is used to construct a composite convolutional U network MCU-Net; the MCU-Net is based on the architecture of the U-net network U-Net, including an input module, an encoder, a decoder and an image reconstruction module; the dilated convolution is introduced into the asymmetric convolution to obtain an asymmetric dilated convolution AD, and AD is used as the input module; multiple residual local feature modules RLFB are used as multi-layer encoders; the RLFB is connected with multi-layer residuals, and the atrous spatial convolution pooling pyramid ASPP is introduced to obtain an enhanced feature aggregation module EFAB, and multiple EFABs corresponding to the RLFB are used as multi-layer decoders.
[0100] The training module 302 is used to train the MCU-Net using structured light image samples containing cells to obtain a cell structured light microscopy reconstruction model.
[0101] The reconstruction module 303 is used to obtain the target cell image, input the target cell image into the cell structure light microscopy reconstruction model, perform preliminary recognition on the input target cell image through the input module to obtain an initial feature map, extract local features of the initial feature map through a multi-layer encoder to obtain an enhanced feature map, perform feature fusion on the enhanced feature map through a multi-layer decoder to obtain a comprehensive feature map, and finally reconstruct and output the comprehensive feature map through the image reconstruction module to obtain a reconstructed target image.
[0102] By using the above-mentioned device, a composite convolutional U network is constructed, and an asymmetric dilated convolution is specifically constructed to realize multi-scale feature extraction, thereby enhancing the effects of scale scaling and primary feature extraction to avoid detail loss in cell images. Furthermore, a residual connection design is realized by constructing an enhanced feature aggregation module, thereby realizing more efficient feature extraction and fusion. While significantly improving the resolution and detail restoration capability of microbial cell images, the computational complexity of related details is further reduced. Not only can the naturalness and continuity of the image be maintained when processing areas rich in details, but also the image can be processed more quickly while maintaining high reconstruction quality, thereby improving the overall reconstruction speed and efficiency, which is conducive to real-time observation of dynamic changes in cells.
[0103] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 The steps of the cell image reconstruction method are provided.
[0104] The present invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The steps of the cell image reconstruction method are provided.
[0105] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0106] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0107] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0109] It should be noted that the above specific implementation method can enable those skilled in the art to understand the invention more comprehensively, but does not limit the invention in any way. Therefore, although the present invention has been described in detail in this specification, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the patent for the invention. Any figure mark in the claims should not be regarded as limiting the claims involved.
Claims
1. A cell image reconstruction method, characterized in that: The method comprises: A composite convolutional U network MCU-Net is constructed; the MCU-Net is based on the architecture of the U-net network, including an input module, an encoder, a decoder and an image reconstruction module; the dilated convolution is introduced into the asymmetric convolution to obtain the asymmetric dilated convolution AD, and the AD is used as the input module; multiple residual local feature modules RLFB are used as multi-layer encoders; the RLFB is connected with multi-layer residuals, and the atrous spatial convolution pooling pyramid ASPP is introduced to obtain the enhanced feature aggregation module EFAB, and the multiple EFABs corresponding to the RLFB are used as multi-layer decoders; the MCU-Net is trained by using the structured light image samples containing cells to obtain a cell structured light microscopy reconstruction model; A target cell image is acquired, and the target cell image is input into a cell structure light microscopy reconstruction model. The input target cell image is preliminarily identified by the input module to obtain an initial feature map. Local features of the initial feature map are extracted by a multi-layer encoder to obtain an enhanced feature map. The enhanced feature map is feature-fused by a multi-layer decoder to obtain a comprehensive feature map. Finally, the comprehensive feature map is reconstructed and output by an image reconstruction module to obtain a reconstructed target image.
2. A cell image reconstruction method according to claim 1, characterized in that: The step of introducing the dilated convolution into the asymmetric convolution to obtain the asymmetric dilated convolution AD comprises: Add dilation factor between kernel elements of asymmetric convolution , get the vertical convolution for: ; Horizontal Convolution for: ; Square Convolution for: ; The vertical convolution, horizontal convolution and square convolution are subjected to feature fusion to obtain the asymmetric dilated convolution AD: ; Among them, the convolution kernel size is , Represents the number of units that the convolution kernel extends outward from the center point, Represents the input feature map In the center The position offset is and The pixel feature value of , and They represent convolution kernels that extend vertically, horizontally, and square convolution kernels, respectively. represents the location of the convolution kernel weight, b Represents a bias term.
3. A cell image reconstruction method according to claim 1, characterized in that: The method of using the plurality of residual local feature modules RLFB as a multi-layer encoder to extract local features of the initial feature map to obtain an enhanced feature map includes: The initial feature map The convolution kernel is The convolution operation is then applied to Re LU Activation function gets feature map ; And the feature map after multiple RLFB input Perform residual connection and get output ; Through the spatial attention module ESA Perform feature transformation And apply the activation function , generate attention weights, and then compare the attention weights with Multiply element by element to get the enhanced feature map .
4. A cell image reconstruction method according to claim 1, characterized in that: The multi-layer residual connection of RLFB includes: The first The RLFB module performs the previous layer feature map Perform feature extraction and obtain the output feature map ; Will With the previous layer feature map And the feature maps of all previous layers Perform residual connection to generate updated feature map .
5. A cell image reconstruction method according to claim 1, characterized in that: The introduction of the hole space convolution pooling pyramid ASPP includes: Each convolution output of the RLFB after the residual connection is concatenated with the output of the previous layer in the channel dimension and further fused through 1x1 convolution.
6. A cell image reconstruction method according to claim 1, characterized in that: The structured light image samples include a low-resolution structured light image and its corresponding super-resolution structured light image; The MCU-Net is trained by using the structured light image samples containing cells to obtain the cell structured light microscopic reconstruction model, including: The low-resolution structured light image is input into MCU-Net to obtain a predicted high-resolution image. The MCU-Net is trained with the goal of minimizing the loss function between the predicted high-resolution image and the corresponding super-resolution structured light image to obtain a cell structured light microscopy reconstruction model.
7. A cell image reconstruction method according to claim 6, characterized in that: By taking the mean square error MSE and structural similarity index SSIM Combined, the loss function is constructed: ; in, is the ratio coefficient of the structural correlation index, used to balance SSIM and MSE The relative contribution of S T and S P represent the true value and the model predicted value respectively.
8. A cell image reconstruction device, characterized in that: The device comprises: A construction module is used to construct a composite convolutional U network MCU-Net; the MCU-Net is based on the architecture of the U-net network, including an input module, an encoder, a decoder and an image reconstruction module; the dilated convolution is introduced into the asymmetric convolution to obtain an asymmetric dilated convolution AD, and the AD is used as the input module; multiple residual local feature modules RLFB are used as multi-layer encoders; the RLFB is connected with multiple layers of residuals, and the hole space convolution pooling pyramid ASPP is introduced to obtain an enhanced feature aggregation module EFAB, and multiple EFABs corresponding to the RLFB are used as multi-layer decoders; A training module is used to train MCU-Net using structured light image samples containing cells to obtain a cell structured light microscopy reconstruction model; A reconstruction module is used to obtain a target cell image, input the target cell image into a cell structure light microscopy reconstruction model, perform preliminary recognition on the input target cell image through the input module to obtain an initial feature map, extract local features of the initial feature map through a multi-layer encoder to obtain an enhanced feature map, perform feature fusion on the enhanced feature map through a multi-layer decoder to obtain a comprehensive feature map, and finally reconstruct and output the comprehensive feature map through an image reconstruction module to obtain a reconstructed target image.
9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
10. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.
Citation Information
Patent Citations
SAR image complex building extraction method and system based on selection attention network
CN111860233A
Image rain removing method and system of multidimensional aggregation network based on rain line feature reconstruction
CN114549334A
