A single polymer growth precise positioning method and system based on a hybrid network
By adopting a two-stage collaborative processing mechanism based on a hybrid network, combining the CS_Diff layer and the CNN-ViT hybrid model, the problems of dynamic fuzzy interference and low texture robustness in single polymer growth monitoring are solved, achieving a unity of nanoscale precision and real-time processing, and meeting the in-situ monitoring needs of polymer materials.
Patent Information
- Application Number
- CN202511202730.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-27
- Publication Date
- 2025-11-11
- Estimated Expiration
- 2045-08-27
AI Technical Summary
Existing technologies for monitoring the growth of single polymers suffer from problems such as dynamic fuzzy interference, poor robustness to low texture, low computational efficiency, insufficient feature coupling of deep learning models, and lack of long-range dependencies, making it difficult to achieve a balance between nanometer-level precision and real-time processing.
A hybrid network-based approach is adopted, which integrates physical and data-driven approaches to construct a two-stage collaborative processing mechanism for focused image generation and defocus depth calculation. The CS_Diff layer is used to enhance the robustness of low-texture scenes, and a CNN-ViT hybrid model is combined to perform multi-scale feature collaborative processing, thereby achieving the unification of nanometer-level axial positioning accuracy and real-time processing.
It achieves a combination of nanometer-level axial positioning accuracy (±10 nm) and high frame rate real-time processing (120 fps), meeting the needs of in-situ monitoring of polymer materials and breaking through the limitations of traditional DFD technology.
Smart Images

Figure CN120708219B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of deep learning, computer vision, and optical measurement technology, and in particular to a method and system for precise localization of single polymer growth based on hybrid networks. Background Technology
[0002] In-situ dynamic monitoring of polymer materials at the nanoscale is a key challenge in materials science and biomedical engineering. Real-time, precise localization of the growth height of single polymers is crucial for a deeper understanding of molecular self-assembly mechanisms and precise control of material properties. However, traditional optical microscopy is limited by the optical diffraction limit, with axial resolution typically only reaching the hundreds of nanometers level, which is insufficient to meet the precision requirements for real-time monitoring of nanoscale growth processes. To overcome this limitation, the depth from defocus (DFD) method has been proposed. This method inverts the object height by analyzing image blur, providing a potential pathway for achieving super-resolution localization.
[0003] Nevertheless, existing DFD technology still faces significant technical bottlenecks when applied to single polymer growth monitoring. First, polymer growth is a continuous dynamic process, and the static defocus model relied upon by traditional DFD is difficult to adapt to rapid deformation, leading to the accumulation of motion artifacts and affecting positioning accuracy. Second, polymer chain surfaces often exhibit weak optical texture features. In low-texture areas, the defocus difference signal is weak and difficult to capture effectively, and existing algorithms (such as gradient detection methods based on the Laplacian operator) are extremely sensitive to such noise. Furthermore, traditional computational methods relying on iterative optimization (such as blind deconvolution) are highly complex and cannot meet the real-time processing requirements of over 100 fps needed for in-situ monitoring.
[0004] In recent years, deep learning models (such as U-Net and ResNet) have been introduced into the field of DFD to improve performance. However, these models still have significant shortcomings when dealing with the specific scenario of single polymer growth. Existing end-to-end networks typically regress the defocus depth value directly, ignoring the inherent physical correlation between the focused and defocused images, leading to the loss of high-frequency detail information and affecting nanoscale accuracy. At the same time, convolutional neural networks (CNNs), due to their inherent local receptive field characteristics, have difficulty effectively modeling the long-range spatial dependence of polymer growth direction, and are prone to directional positioning bias. In addition, multi-scale diffraction features generated during growth (such as primary diffraction spots and secondary interference rings) are easily attenuated during transmission in deep networks, and existing cross-layer connection mechanisms have failed to adequately solve the problem of effective preservation and fusion of cross-scale nanoscale signals.
[0005] Therefore, there is an urgent need for an innovative solution that can effectively address issues such as dynamic blur interference, poor robustness of low-texture models, low computational efficiency, insufficient feature coupling, lack of long-range dependencies, and cross-scale feature fragmentation in existing deep learning models. Summary of the Invention
[0006] The problem to be solved by this invention is to provide a method and system for precise positioning of single polymer growth based on a hybrid network. Through an innovative design that integrates physical and data-driven approaches, a two-stage collaborative processing mechanism for focused image generation and defocus depth calculation is constructed to achieve the unification of nanometer-level axial positioning accuracy and real-time processing.
[0007] This invention adopts the following technical solution: a method for precise localization of single polymer growth based on a hybrid network, comprising the following steps:
[0008] Pre-focusing stage:
[0009] S101. Obtain the original defocused image DF_map of single polymer growth, and obtain the corresponding real focused image GT_map by adjusting the microscope focusing structure.
[0010] S102. Based on the PreFocusNet model, the structure similarity map DFOut_map between the original defocused image DF_map and the generated focused image Out_map, as well as the structure similarity map DFGT_map between the original defocused image DF_map and the real focused image GT_map, are obtained using the CS_Diff layer.
[0011] S103. Based on the obtained structural similarity maps DFOut_map and DFGT_map, calculate the structural difference features;
[0012] S104. Optimize the PreFocusNet model using the structural difference features to generate a high-precision focused image Out_map1;
[0013] Positioning phase:
[0014] S201. Fix the PreFocusNet model parameters and use the CS_Diff layer again to obtain the structural similarity map DFOut_map1 between the original defocused image DF_map and the high-precision focused image Out_map1.
[0015] S202. The original defocused image DF_map and the structural similarity map DFOut_map1 are fused to form a dual-channel fused feature map;
[0016] S203. Input the dual-channel fused feature map into the CNN-ViT hybrid model and perform three-level processing to obtain the nanometer-level defocus depth value of the single polymer growth position.
[0017] Preferably, in step S103, a pixel-by-pixel difference analysis is performed on DFOut_map and DFGT_map to extract structural difference features. This is used to guide the PreFocusNet model to minimize the perceptual difference between the generated focused image Out_map and the real focused image GT_map during training.
[0018] Preferably, in step S104, a high-precision focused image is generated, and a perceptual loss is defined, which is achieved by minimizing the loss. Perform backpropagation to update the PreFocusNet parameters until convergence, and obtain a high-precision focusing model.
[0019] Preferably, in step S202, the fusion method is channel stitching, where the original out-of-focus image DF_map and the structural similarity map DFOut_map1 are stitched together to generate a dual-channel fused input with an input size of [size missing]. ,in, These represent the height and width of the fused feature map, respectively.
[0020] Preferably, in step S203, the dual-channel fused input is fed into the CNN-ViT hybrid model for three-level processing, as follows:
[0021] First-level processing: The first part of the CNN-ViT hybrid model includes two sets of Double Conv layers and MaxPool layers to extract primary diffraction features. The convolution kernel size is 3×3. Each Conv is followed by BatchNorm and ReLU activation. The pooling kernel size is 2×2 with a stride of 2.
[0022] Second-level processing: The second part of the CNN-ViT hybrid model combines the linear embedding layer and fast mapping mechanism of the visual Transformer module to analyze the long-range spatial correlation features of the growth direction of a single polymer.
[0023] The third-level processing: The third part of the CNN-ViT hybrid model introduces a cross-layer skip connection mechanism, which integrates the intermediate features of the first-level and second-level processing, and outputs the nanometer-scale defocus depth value of the predicted single polymer growth position.
[0024] Preferably, in the second-level processing, the long-range spatial correlation characteristics of the growth direction of the single polymer are analyzed, and the method is as follows:
[0025] The input convolutional feature map is patched by the linear embedding layer of the Transformer module and flattened into a one-dimensional vector. It is then linearly mapped to d-dimensional space through a fully connected layer to obtain the Token sequence.
[0026] The token sequence is input into the attention module to construct query, key, and value vectors, and the attention matrix is calculated based on relative position weights. The attention outputs of different heads are concatenated and linearly mapped through a multi-head self-attention mechanism to obtain an updated feature sequence containing global spatial dependencies.
[0027] The updated feature sequence is inversely transformed into a two-dimensional feature map format according to the Patch partitioning rule, and feature fusion is performed. A fast mapping mechanism is used to capture the long-range spatial correlation features of the growth direction of a single polymer, resulting in a long-range spatial correlation feature map.
[0028] Preferably, in the third-level processing, the cross-layer skip connection mechanism directly connects the primary diffraction features output from the first-level processing to the input of the third-level processing, and fuses them with the long-range spatial correlation features output from the second-level processing in the channel dimension, thereby enhancing feature reuse and gradient propagation, suppressing the decay of deep network features, and obtaining the final fused feature map.
[0029] Preferably, the positioning method has an axial positioning accuracy of ±10 nanometers and a processing speed of 120 frames per second or more.
[0030] The present invention also provides: a single polymer growth precise positioning system based on a hybrid network, used to implement any of the above-mentioned single polymer growth precise positioning methods, including: an image acquisition module, a pre-focusing processing module, and a positioning processing module;
[0031] The image acquisition module is used to acquire the original defocused image DF_map of single polymer growth and the corresponding real focused image GT_map;
[0032] The prefocusing processing module, connected to the image acquisition module, is used to perform prefocusing operations, including:
[0033] CS_Diff unit: Extracts the structural similarity map DFOut_map between the original defocused image DF_map and the generated focused image Out_map, and the structural similarity map DFGT_map between the original defocused image DF_map and the real focused image GT_map, and calculates the structural difference features between the generated focused image and the real focused image;
[0034] PreFocusNet unit: Receives the original defocused image DF_map, optimizes it using the structural difference features calculated by the CS_Diff unit, and outputs a high-precision focused image Out_map1.
[0035] The positioning processing module, connected to the pre-focusing processing module, is used to perform defocus depth calculation, including:
[0036] Structural similarity map generation unit: The CS_Diff unit is used to calculate the difference between the high-precision focused image Out_map1 and the original defocused image DF_map to generate a structural similarity map DFOut_map1;
[0037] Fusion unit: used to fuse the original defocused image and the structural similarity map DFOut_map1 to form a dual-channel fused feature map;
[0038] CNN-ViT hybrid processing unit: used to receive the dual-channel fused feature map, and sequentially perform three-level processing to output the nanometer-level defocus depth value of the single polymer growth position.
[0039] Preferably, the CNN-ViT hybrid processing unit includes:
[0040] First-level processing subunit: Extracts primary diffraction features through Double Conv layer and MaxPool layer;
[0041] The second-level processing subunit combines the Linear Embedding layer and Faster Mapping mechanism of the ViT module to analyze the long-range spatial correlation features of the growth direction;
[0042] The third-level processing subunit introduces cross-layer skip connections, integrates features from the first-level and second-level processing subunits, enhances feature reuse, and outputs nanometer-scale defocus depth values for the single polymer growth location.
[0043] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:
[0044] 1. Physical-Data Dual-Driven Architecture: The method of this invention incorporates the physical prior of SSIM into deep learning through the CS_Diff layer, thereby enhancing the robustness of low-texture scenes.
[0045] 2. Dual-channel information complementarity: The method of this invention retains blur information and focus differences while fusing the original defocused image with the Diff-Map, thus breaking through the feature coupling bottleneck.
[0046] 3. Multi-scale feature collaboration: The CNN-ViT hybrid model of this invention combines local diffraction feature extraction, global long-range dependency modeling and cross-layer feature reuse to achieve full-scale capture of nanoscale signals.
[0047] 4. Real-time high-performance output: The processing flow of the present invention adopts a lightweight design, which supports real-time processing of 120 fps while ensuring axial positioning accuracy of ±10 nanometers, meeting the stringent requirements of in-situ monitoring of polymer materials. Attached Figure Description
[0048] Figure 1 This is a flowchart illustrating the precise positioning process for single polymer growth according to the present invention.
[0049] Figure 2 This is a diagram illustrating the processing steps of the training dataset obtained in an embodiment of the present invention.
[0050] Figure 3 This is a diagram of the model structure of PreFocusNet in this invention;
[0051] Figure 4 This is a set of defocus-focus image pairs and structural similarity diagrams according to an embodiment of the present invention;
[0052] Figure 5 This is a structural diagram of the CNN-ViT hybrid model of the present invention;
[0053] Figure 6 This is a schematic diagram of the test results of the simulated dataset in an embodiment of the present invention;
[0054] Figure 7 This is a schematic diagram of the test results of a single polymer growth dataset in an embodiment of the present invention. Detailed Implementation
[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the application will be further described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in this invention. All non-innovative embodiments based on these embodiments by other researchers in the art are within the protection scope of this invention. Furthermore, the step numbers in the embodiments of this invention are only set for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.
[0056] Example 1
[0057] A precise positioning system for single polymer growth based on a hybrid network is proposed, such as... Figure 1 As shown, it includes an image acquisition module, a prefocusing processing module, and a positioning processing module, and the algorithm pipeline is optimized through hardware acceleration.
[0058] The image acquisition module is configured to acquire raw defocused images of single polymer growth.
[0059] The prefocusing processing module is connected to the image acquisition module and is configured to perform prefocusing operations. It integrates the CS_Diff unit and the PreFocusNet unit, which are used to extract structural similarity maps and optimize the focused image, respectively.
[0060] The CS_Diff unit is configured to extract the structural similarity map DFOut_map between the original defocused image DF_map and the generated focused image Out_map, as well as the structural similarity map DFGT_map between DF_map and the real focused image GT_map, and to calculate the difference features between the generated focused image Out_map and the real focused image GT_map based on these.
[0061] PreFocusNet unit: configured to receive the original out-of-focus image and optimize it using the difference features calculated by the CS_Diff unit to output a high-precision focused image.
[0062] Positioning processing module: Connected to the prefocusing processing module, configured to perform defocus depth calculation, including: difference map generation unit, dual-channel fusion unit, CNN-ViT hybrid processing unit and depth output unit.
[0063] Among them, the differential image generation unit is configured to use the CS_Diff unit to calculate the difference between the high-precision focused image generated by the PreFocusNet unit and the original defocused image DF_map, and generate the difference map DFOut_map1.
[0064] Fusion unit: configured to fuse the original out-of-focus image and the difference map DFOut_map1 to form a dual-channel input.
[0065] CNN-ViT Hybrid Processing Unit: Configured to receive the dual-channel input, internally sets up and sequentially executes the first-level subunit, the second-level subunit, and the third-level subunit.
[0066] First-level processing subunit: Extracts primary diffraction features through Double Conv layer and MaxPool layer;
[0067] The second-level processing subunit combines the Linear Embedding layer and Faster Mapping mechanism of the ViT module to analyze the long-range spatial correlation features of the growth direction;
[0068] The third-level processing subunit introduces cross-layer skip connections, which bring the output of the first-level processing subunit into its own input. It integrates features from the first-level and second-level processing subunits, enhances feature reuse, and outputs nanometer-scale defocus depth values at the growth position of a single polymer.
[0069] Example 2
[0070] A precise localization method for single polymer growth based on hybrid networks is proposed, and a two-stage processing flow is designed, including a pre-focusing stage and a localization stage.
[0071] In this implementation, the experimental setup and environment were as follows: an image acquisition system based on an optical microscope and a CMOS camera was used, with a camera resolution of 1024×768 pixels and a pixel size of 3.45μm×3.45μm; the microscope objective was a 60×x high numerical aperture (NA=0.85) oil immersion objective.
[0072] In this embodiment, the pre-focusing stage is specifically processed as follows:
[0073] Step S101: The image acquisition module uses the microscope focusing adjustment structure and piezoelectric ceramic (PZT) driving system to... The step size is scanned along the axial direction (z direction), such as Figure 2 As shown, the multi-plane defocused image sequence DF_map corresponding to the polystyrene microspheres in the single polymer growth region is obtained and represented as:
[0074] ;
[0075] in, As the initial reference focal plane position, Defocused images at different depths The number of scanning layers within the range of ±200nm.
[0076] Then, defocused images at different depths were analyzed. The center point of a single polystyrene microsphere is manually located, and the microsphere is cut into 128×128 pixel blocks from the center to obtain 500 sets of sub-image blocks for subsequent network training.
[0077] ;
[0078] in, Represents the sequence of small blocks after pruning, ( () represents the center coordinates of the located individual microsphere. Indicates ( Extract a 128×128 image patch centered on the image. express The width, height, and depth dimensions of the corresponding image.
[0079] Step S102: Cut the small block sequence Input the PreFocusNet model for training. The structure of PreFocusNet is as follows: Figure 3 As shown.
[0080] In this embodiment, the PreFocusNet model adopts an encoder-decoder architecture to process the cropped 128×128 convolutions (ReLU activation) and 2×2 max pooling operations, gradually increasing the number of channels to 1024, and reducing the feature map size to 8×8; in the decoding stage, the size is restored to 128×128 through 2×2 upsampling and residual connections, and the features are refined with 1×1 convolutions, and finally the output layer generates a high-dimensional feature map.
[0081] The PreFocusNet model training process uses the CS_Diff layer to calculate and generate a structural similarity map DFOut_map between the focused image Out_map and the DF sequence, and a structural similarity map DFGT_map between the real focused image GT_map and the DF sequence, as shown below. Figure 4 As shown, the specific formula is as follows:
[0082] ;
[0083] ;
[0084] in, Represents the original defocused image With generating focused images Structural similarity diagram, Represents the original defocused image With true focus image The structural similarity diagram.
[0085] Specifically, the CS_Diff layer is used to generate a structural similarity graph, as follows:
[0086] ;
[0087] in, The input image for the CS_Diff layer. express covariance, and They represent variance It is a constant.
[0088] In this embodiment, preferably, To represent a small constant, usually, and It is used for numerical stability, and its function is to prevent the denominator from being zero.
[0089] Step S103: Perform pixel-by-pixel difference analysis on DFOut_map and DFGT_map to extract structural difference features:
[0090] ;
[0091] Step S104: Define the perceptual loss as follows:
[0092] ;
[0093] in, This represents the pre-trained feature mapping of the PreFocusNet model. , These represent the generated focused image and the true focused image, respectively.
[0094] By minimizing the loss Perform backpropagation to update the PreFocusNet parameters until convergence, and obtain a high-precision focusing model.
[0095] Furthermore, in this embodiment, the positioning stage is specifically processed as follows:
[0096] Step S201: Freeze the parameters of the trained PreFocusNet model, and use the CS_Diff layer again to calculate the structural similarity map DFOut_map between DF_map and Out_map.
[0097] Step S202: Concatenate the original out-of-focus image DF_map and the DFOut_map to generate a dual-channel fused feature map. The input image size is 2×H×W, where... These represent the height and width of the fused feature map, respectively.
[0098] Step S203: Input the dual-channel fused feature map into the CNN-ViT hybrid model for three-level processing. The overall structure is as follows: Figure 5 As shown.
[0099] Step S203a (first-level processing): The first part of the CNN-ViT hybrid model consists of two sets of double convolutional layers (DoubleConv) and max pooling layers (MaxPool) used to extract primary diffraction features.
[0100] In this embodiment, preferably, the convolution kernel size is 3×3, each Conv group is followed by BatchNorm and ReLU activation, the pooling kernel size is 2×2, and the stride is 2.
[0101] The initial input size is H=W=128, and the size is halved after each pooling round. The output feature map after three rounds of pooling is:
[0102] .
[0103] Preferably, =64, This is the primary diffraction feature map output from the first stage of processing.
[0104] Step S203b (Second-level processing): Combining the linear embedding layer and fast mapping mechanism of the Visual Transformer (ViT) module, the long-range spatial correlation features of the polymer growth direction are analyzed to solve the modeling limitations of the local receptive field of traditional CNNs.
[0105] In this embodiment, the dual-channel fusion feature map from step S202 is linearly mapped to a ViT Token sequence through a Linear Embedding layer, and a Faster Mapping mechanism (i.e., multi-head self-attention combined with relative position encoding) is used to capture the long-range spatial correlation features of the growth direction of a single polymer.
[0106] Specifically, to adapt the feature map to the Transformer module, the convolutional feature map is first patched and flattened through the Linear Embedding layer and linearly mapped to a token sequence.
[0107] The input feature map is divided into several non-overlapping patch regions. Assuming the patch size is P×P, for example, P=4, it can be divided into... Each patch is flattened into a one-dimensional vector and linearly mapped to a d-dimensional space through a fully connected layer to obtain a token sequence. .
[0108] In one alternative, the embedding dimension d is set to 64.
[0109] To enhance the expressive power of features in the global spatial dimension, the token sequence is further input into the improved Transformer module, which includes a multi-head self-attention mechanism for parallel modeling of spatial dependencies in different subspaces and a relative positional encoding mechanism to enhance the model's ability to perceive spatial structure.
[0110] Preferably, the attention module's processing includes: constructing query, key, and value vectors; calculating an attention matrix based on relative position weights; and concatenating and linearly mapping the attention outputs from different heads. Through this mechanism, an updated feature sequence containing global spatial dependencies can be obtained. .
[0111] The updated feature sequence It can be inversely transformed into a two-dimensional feature map format according to the patch partitioning rules for subsequent feature fusion processing. That is, it is reconstructed as:
[0112] ;
[0113] in, This represents the output feature map that incorporates long-range spatial dependency information.
[0114] Step S203c (Third-level processing): A cross-layer skip connection mechanism is introduced, directly connecting the primary diffraction features output from the first-level processing to the entry point of the third-level processing. This is then fused with the long-range correlation features output from the second-level processing along the channel dimension to enhance feature reuse and gradient propagation, effectively suppressing feature decay in deep networks, and ultimately obtaining a fused feature map.
[0115] ;
[0116] in, The operator refers to merging two feature maps along the channel dimension (0th dimension).
[0117] Step S204: The final output of the CNN-ViT hybrid model is used to obtain the nanometer-level defocus depth value of the single polymer growth position through the output of the fully connected layer, with a positioning accuracy of ±10 nanometers.
[0118] Furthermore, based on the method of this invention, a dataset was built in the laboratory. After 100 epochs of training, the test frame rate after convergence reached over 120 frames per second (fps), meeting the real-time requirements; Figure 6 As shown, the axial positioning accuracy evaluation results obtained using a simulated dataset show an average error of less than 8 nanometers and a standard deviation of less than 5 nanometers. The trained model is then applied to in-situ growth, as... Figure 7 As shown, the depolymerization and polymerization processes during the growth process can be accurately reproduced.
[0119] In summary, the single polymer growth precision localization method and system based on hybrid networks proposed in this invention innovatively integrates a two-stage collaborative processing flow of focused image generation and defocus depth calculation. At the same time, it combines a physics-inspired CS_Diff layer design and a multi-level feature parsing mechanism of the CNN-ViT hybrid model, which can achieve a breakthrough balance between nanometer-level axial positioning accuracy (±10 nm) and high frame rate real-time processing (120 fps), thus providing strong technical support for in-situ dynamic monitoring of polymer materials.
[0120] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A method for precise localization of single polymer growth based on hybrid networks, characterized in that, Includes the following steps: Pre-focusing stage: S101. Obtain the original defocused image DF_map of single polymer growth, and obtain the corresponding real focused image GT_map by adjusting the microscope focusing structure. S102. Based on the PreFocusNet model, the structure similarity map DFOut_map between the original defocused image DF_map and the generated focused image Out_map, as well as the structure similarity map DFGT_map between the original defocused image DF_map and the real focused image GT_map, are obtained using the CS_Diff layer. The PreFocusNet model adopts an encoder-decoder architecture to process the cropped 128×128 convolutions and 2×2 max pooling operations, gradually increasing the number of channels to 1024, and reducing the feature map size to 8×8; in the decoding stage, the size is restored to 128×128 through 2×2 upsampling and residual connections, and the features are refined with 1×1 convolutions, and the output layer generates a high-dimensional feature map. S103. Based on the obtained structural similarity maps DFOut_map and DFGT_map, calculate the structural difference features; S104. Optimize the PreFocusNet model using the structural difference features to generate a high-precision focused image Out_map1; Positioning phase: S201. Fix the PreFocusNet model parameters and use the CS_Diff layer again to obtain the structural similarity map DFOut_map1 between the original defocused image DF_map and the high-precision focused image Out_map1. S202. The original defocused image DF_map and the structural similarity map DFOut_map1 are fused to form a dual-channel fused feature map; S203. Input the dual-channel fused feature map into the CNN-ViT hybrid model and perform three-level processing to obtain the nanometer-level defocus depth value of the single polymer growth position.
2. The method for precise positioning of single polymer growth based on hybrid networks according to claim 1, characterized in that, In step S102, a structural similarity graph is generated using the CS_Diff layer, as follows: ; in, The input image for the CS_Diff layer. express covariance; and They represent variance It is a constant; ; ; in, Represents the original out-of-focus image With generating focused images Structural similarity diagram, Represents the original defocused image With true focus image The structural similarity diagram.
3. The method for precise positioning of single polymer growth based on hybrid networks according to claim 2, characterized in that, In step S103, pixel-by-pixel difference analysis is performed on DFOut_map and DFGT_map to extract structural difference features, calculated as follows: ; in, The obtained structural difference features are used to guide the PreFocusNet model to minimize the perceptual difference between the generated focused image Out_map and the real focused image GT_map during training.
4. The method for precise positioning of single polymer growth based on hybrid networks according to claim 1, characterized in that, In step S104, a high-precision focused image is generated, as follows: The perceptual loss is defined as follows: ; in, This represents the pre-trained feature maps of the PreFocusNet model. , These represent the generated focused image and the true focused image, respectively. By minimizing the loss Perform backpropagation to update the PreFocusNet parameters until convergence, and obtain a high-precision focusing model.
5. The method for precise positioning of single polymer growth based on hybrid networks according to claim 1, characterized in that, In step S202, the fusion method is channel stitching. The original out-of-focus image DF_map and the structural similarity map DFOut_map1 are stitched together to generate a dual-channel fused input with an input size of [size missing]. ,in, These represent the height and width of the fused feature map, respectively.
6. The method for precise positioning of single polymer growth based on hybrid networks according to claim 5, characterized in that, In step S203, the dual-channel fused input is fed into the CNN-ViT hybrid model for three-level processing, as follows: First-level processing: The first part of the CNN-ViT hybrid model includes two sets of Double Conv layers and MaxPool layers, which are used to extract primary diffraction features. The convolution kernel size is 3×3. Each Conv is followed by BatchNorm and ReLU activation. The pooling kernel size is 2×2 with a stride of 2. The output is the primary diffraction feature map. Second-level processing: The second part of the CNN-ViT hybrid model combines the linear embedding layer and fast mapping mechanism of the visual Transformer module to analyze the long-range spatial correlation features of the growth direction of a single polymer. The third-level processing: The third part of the CNN-ViT hybrid model introduces a cross-layer skip connection mechanism, which integrates the intermediate features of the first-level and second-level processing, and outputs the nanometer-scale defocus depth value of the predicted single polymer growth position.
7. The method for precise positioning of single polymer growth based on hybrid networks according to claim 6, characterized in that, In the second-level processing, the long-range spatial correlation characteristics of the growth direction of a single polymer are analyzed, and the method is as follows: The input convolutional feature map is patched by the linear embedding layer of the Transformer module and flattened into a one-dimensional vector. It is then linearly mapped to d-dimensional space through a fully connected layer to obtain the Token sequence. The token sequence is input into the attention module to construct query, key, and value vectors, and the attention matrix is calculated based on relative position weights. The attention outputs of different heads are concatenated and linearly mapped through a multi-head self-attention mechanism to obtain an updated feature sequence containing global spatial dependencies. The updated feature sequence is inversely transformed into a two-dimensional feature map format according to the Patch partitioning rule, and feature fusion is performed. A fast mapping mechanism is used to capture the long-range spatial correlation features of the growth direction of a single polymer, resulting in a long-range spatial correlation feature map.
8. The method for precise positioning of single polymer growth based on hybrid networks according to claim 7, characterized in that, In the third-level processing, the cross-layer skip connection mechanism directly connects the primary diffraction features output from the first-level processing to the input of the third-level processing, and fuses them with the long-range spatial correlation features output from the second-level processing in the channel dimension, thereby enhancing feature reuse and gradient propagation, suppressing feature decay in deep networks, and obtaining the final fused feature map.
9. The method for precise positioning of single polymer growth based on hybrid networks according to claim 6, characterized in that, The CNN-ViT hybrid model outputs nanometer-level defocus depth values of the predicted single polymer growth location through a fully connected layer, with an axial positioning accuracy of ±10 nanometers and a processing speed of 120 frames per second or more.
10. A single polymer growth precision positioning system based on a hybrid network, used to implement the single polymer growth precision positioning method according to any one of claims 1 to 8, characterized in that, include: Image acquisition module, prefocusing processing module, positioning processing module; The image acquisition module is used to acquire the original defocused image DF_map of single polymer growth and the corresponding real focused image GT_map; The prefocusing processing module, connected to the image acquisition module, is used to perform prefocusing operations, including: CS_Diff unit: Extracts the structural similarity map DFOut_map between the original defocused image DF_map and the generated focused image Out_map, and the structural similarity map DFGT_map between the original defocused image DF_map and the real focused image GT_map, and calculates the structural difference features between the generated focused image and the real focused image; PreFocusNet unit: Receives the original out-of-focus image DF_map, optimizes it using the structural difference features calculated by the CS_Diff unit, and outputs a high-precision focused image Out_map1; The positioning processing module, connected to the pre-focusing processing module, is used to perform defocus depth calculation, including: Structural similarity map generation unit: The CS_Diff unit is used to calculate the difference between the high-precision focused image Out_map1 and the original defocused image DF_map to generate a structural similarity map DFOut_map1; Fusion unit: used to fuse the original defocused image and the structural similarity map DFOut_map1 to form a dual-channel fused feature map; CNN-ViT hybrid processing unit: Used to receive the dual-channel fused feature map and sequentially perform three levels of processing, including: First-level processing subunit: Extracts primary diffraction features through Double Conv layer and MaxPool layer; The second-level processing subunit combines the Linear Embedding layer and Faster Mapping mechanism of the ViT module to analyze the long-range spatial correlation features of the growth direction; The third-level processing subunit introduces cross-layer skip connections, integrates features from the first-level and second-level processing subunits, enhances feature reuse, and outputs nanometer-scale defocus depth values for the single polymer growth location.
Citation Information
Patent Citations
Multi-focus image fusion method based on multi-scale feature interaction network
CN113705675A
Multi-focal-length image fusion model, method and device for compound eye camera
CN115439376A