Deep learning-based microscopic image seamless splicing and enhanced reconstruction method

Abstract

The invention discloses a microscopic image seamless splicing and enhanced reconstruction method based on deep learning, and the method comprises the following steps: S1, collecting a plurality of original images with overlapped regions, and recording the spatial position information and imaging parameters of the original images; s2, preprocessing the original image to generate a standardized image sequence; s3, inputting the standardized image into a structure perception feature extraction network, and extracting a feature map fusing textures and structures; s4, inputting the feature image and the original image into an image registration module; s5, inputting the registration image into the boundary attention splicing network; s6, inputting the seamless image into the residual hierarchy reconstruction network, and enhancing image details through hole convolution and multi-scale branches; s7, image quality evaluation is executed, and the structural similarity, the signal-to-noise ratio and the edge retention rate are calculated; and S8, constructing a training set and carrying out end-to-end training optimization based on a joint loss function. According to the method, a multi-module deep network is fused, and seamless splicing and high-quality enhanced reconstruction of microscopic images are realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent reconstruction technology of digital microscopic images, and in particular to a method for seamless stitching and enhanced reconstruction of microscopic images based on deep learning. Background Technology

[0002] With the rapid growth in demand for digital pathology, cell structure analysis, and high-resolution tissue section modeling, microscopic image stitching and reconstruction technology has become a key research direction in the field of microscopic imaging. Due to the limited field of view of a single microscopic image, the actual acquisition process often requires acquiring a large number of spatially overlapping local field-of-view images and stitching them into a high-resolution, large-format image to meet the integrity and continuity requirements of subsequent tasks such as tissue identification and quantitative analysis. Traditional microscopic image stitching methods mainly rely on registration techniques based on feature point matching and multi-image fusion strategies. Typical methods include image alignment algorithms based on local descriptors such as SIFT, SURF, and ORB, as well as image stitching algorithms based on weighted averaging, pyramid fusion, or minimum gap path optimization.

[0003] However, under complex sample conditions, such as significant tissue deformation, large variations in the imaging environment, discontinuous textures in boundary regions, or varying degrees of image blur, traditional stitching algorithms are prone to structural misalignment, abrupt seam boundaries, brightness jumps, and detail loss in overlapping image areas, severely impacting the overall usability and visual consistency of the image. On the other hand, conventional reconstruction and enhancement algorithms primarily rely on filtering or interpolation, making it difficult to effectively recover the fine textures in tissue structures, especially in seam regions where blurring, distortion, and even structural breakage are common. Existing methods generally lack a collaborative optimization mechanism for the entire process of microscopic image registration, seamless fusion, and high-quality reconstruction under an end-to-end unified architecture, particularly lacking the ability to jointly model image structural continuity and boundary geometric consistency.

[0004] In recent years, with the widespread application of deep learning in image generation and super-resolution, some studies have attempted to apply convolutional neural networks to microscopic image stitching and reconstruction tasks, achieving certain image enhancement effects. However, most existing neural network methods focus on image reconstruction itself and fail to systematically address image registration and boundary fusion issues. In practical applications, they still face technical obstacles such as insufficient accuracy, unstable stitching, blurred edges, and training non-convergence. Furthermore, there is a lack of effective modeling methods for microscopic image characteristics such as structural deformation, image degradation, and multi-scale texture coexistence, resulting in limited adaptability and generalization ability of existing models.

[0005] Based on the above problems, there is an urgent need to propose a deep learning method with adaptive spatial registration capability, boundary-aware fusion mechanism and detail enhancement reconstruction strategy, which can achieve unified modeling and end-to-end optimization processing in the process of complex deformation of microscopic images, multi-image stitching and quality improvement, thereby improving the overall structural continuity, stitching naturalness and detail clarity of the image.

[0006] Therefore, how to provide a method for seamless stitching and enhanced reconstruction of microscopic images based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] One objective of this invention is to propose a method for seamless stitching and enhanced reconstruction of microscopic images based on deep learning. This invention constructs a deep learning model consisting of modules for structure perception, spatial alignment, boundary fusion, and multi-scale reconstruction, which enables seamless stitching and detail enhancement of microscopic images under conditions of non-rigid deformation and fuzzy interference. This solves the problems of abrupt seams, structural breaks, and unclear images in existing technologies, thereby improving the overall image quality and structural continuity.

[0008] A method for seamless stitching and enhanced reconstruction of microscopic images based on deep learning according to an embodiment of the present invention includes the following steps: S1. Acquire multiple original images with spatially overlapping areas obtained by a microscopic imaging device, and record the spatial location information and imaging parameters of each image; S2. Preprocess the original image to obtain a standardized image sequence; S3. Input the standardized image sequence into a structure-aware feature extraction network, which includes a deformable convolutional coding module and a channel selection module; S4. Input the feature map and the original image into the image registration module. The image registration module includes a local response locator, an affine parameter estimator, and a bilinear interpolation reconstructor. S5. Input the registered image sequence into a boundary attention-driven stitching network, which includes a boundary saliency detection module, a region weight guidance module, and a continuous structure completion module, and output a seamless image; S6. Input the seamless image into the residual hierarchical reconstruction network, which includes multiple dilated convolution stacked modules and multi-scale reconstruction branches, and outputs an enhanced reconstructed image. S7. Perform quality assessment on the enhanced reconstructed image, including calculating the structural similarity index, peak signal-to-noise ratio, and edge preservation rate; S8. Construct a training sample image set and perform end-to-end joint training using a joint loss optimization mechanism.

[0009] Optionally, the preprocessing includes brightness normalization, edge-preserving filtering, gamma correction, and image color consistency adjustment.

[0010] Optionally, S3 specifically includes: S31. Input the standardized image sequence into the input end of the structure-aware feature extraction network and map it to the initial feature tensor space to preserve the original spatial arrangement structure of the image. S32. In the first encoding stage of the structure-aware feature extraction network, a regular convolutional layer with a fixed receptive field is used to downsample the input image and perform batch normalization and activation function processing to obtain the basic low-level texture response map. S33. Input the low-level texture response map into the deformable convolutional coding module. The deformable convolutional coding module includes multiple deformable convolutional units. Each deformable convolutional unit introduces a learnable offset parameter to adjust the position of the standard sampling grid, thereby achieving adaptive modeling of the curved, non-rigid deformation and skewed areas of the target edge. S34. Concatenate the feature maps output by multiple deformable convolutional layers to form a composite texture encoding map containing different perceptual scales and response directions, and maintain the original spatial information without loss through residual connections. S35. The above composite texture encoding map is sent to the channel selection module. The channel selection module constructs multiple parallel convolution paths. Each convolution path uses a convolution kernel of a different size to form a multi-scale response structure for different receptive field scales. S36. In the channel selection module, a global average pooling operation is performed on the output of each path to calculate the response statistics of each channel. A channel attention mechanism is introduced to generate channel weighting coefficients. Based on the weighting result, the multi-path convolution outputs are fused to obtain a mid-to-high-level semantic feature map with the ability to adapt to differences in tissue density. S37. Input the fused mid-to-high-level feature map into the multi-branch stacking module. The multi-branch stacking module is composed of multiple deformable convolutional and channel selection units cascaded together. It enhances the encoding ability of complex texture transitions, boundary structures and micro-deformation regions by layer-by-layer enhancement. S38. Perform spatial alignment, feature normalization, and dimension fusion on the final output of all stacked units to output a high-dimensional feature tensor in a unified format. This feature tensor retains both the local texture details and global geometric structure information of the image, which is then used as input for the image registration module.

[0011] Optionally, S4 specifically includes: S41. The high-dimensional feature map output by the structure-aware feature extraction network is concatenated and fused with the corresponding original image in the channel dimension to form an input tensor, which is then input into the image registration module. S42. In the image registration module, the local response locator scans the local region of the fused image to extract high response features in the overlapping region of the image, including significant edges, texture abrupt changes and alignment-sensitive regions, and outputs a local response heatmap. S43. Input the local response heatmap into the affine parameter estimator, which consists of a convolutional layer and a fully connected layer. Extract the geometric transformation pattern from the local response and output six affine transformation parameters, including horizontal displacement, vertical displacement, rotation angle, horizontal and vertical scaling factors and shearing factor. S44. Generate the corresponding two-dimensional affine transformation matrix according to the affine parameters, and construct the transformation mapping field in the target coordinate space; S45. Input the original image and the affine transformation matrix into a bilinear interpolation reconstructor. The bilinear interpolation reconstructor performs continuous sampling on the mapped coordinates and remaps the pixels of the original image according to the bilinear interpolation rules to generate an aligned image. S46. Apply the above registration operation to each pair of adjacent images, and repeat the local response extraction, affine parameter estimation and interpolation reconstruction process to output the registered image sequence. S47. Perform structural alignment detection on each overlapping region in the output image sequence, including contour closure, edge consistency and pixel drift error statistics. S48. If the structure alignment detection does not meet the preset threshold condition, the current image pair is re-input into the affine parameter estimator to update the transformation parameters, and the interpolation reconstruction operation is repeated until the condition is met, and finally a spatially consistent registered image sequence is formed.

[0012] Optionally, S5 specifically includes: S51. Input the registered image sequence output by the image registration module into the boundary attention-driven stitching network in spatial order to initialize the image stitching process. S52. The response analysis of the overlapping area of ​​the image is performed by the boundary saliency detection module. The potential splicing seam area is extracted by the method based on local gradient statistics and texture difference mapping, and a saliency boundary heatmap is generated. S53. The salient boundary heatmap and the feature map of the original image are fused and input into the region weight guidance module. The region weight guidance module constructs a fusion priority map and calculates the image fusion weight based on the brightness balance, texture integrity and boundary continuity between the overlapping images. S54. Inside the region weight guidance module, different fusion weights are assigned to local regions of multiple overlapping images through an attention mechanism to form a weighted fusion image candidate set, and conflict regions are masked in the fusion image while retaining stable boundary regions. S55. The candidate image fusion result is sent to the continuous structure completion module to perform boundary repair operation. The continuous structure completion module is composed of multiple residual convolution units to perform contextual texture compensation and structural symmetry restoration on the structural interruption area at the splicing boundary. S56. In the continuous structure completion module, the feature information of adjacent non-seamless regions is diffused to the splicing boundary along the spatial gradient direction through the texture propagation mechanism to fill the boundary discontinuities and generate a fused image with continuous structure. S57. Perform a spatial consistency check on the stitched image to determine whether the boundary transition area meets the conditions of brightness consistency, contour closure and texture smoothness. S58. Output the seamless image after the above boundary detection, region weighting, and crack repair processing. The stitched image has smooth boundaries, continuous tissue structure, and complete texture details, and has a seamless stitching effect. It serves as the input for the subsequent image enhancement and reconstruction network.

[0013] Optionally, S6 specifically includes: S61. The seamless image output by the stitching network is fed into the input end of the residual hierarchical reconstruction network as the input image to initialize the image enhancement and reconstruction process. S62. In the initial module of the reconstruction network, standard convolutional coding operations are performed to extract shallow features of the image and perform normalization processing, providing a structural basis for subsequent multilayer perception. S63. Input shallow features into the dilated convolution stacking module, which consists of multiple dilated convolution layers with different dilation rates, to expand the receptive field while maintaining the resolution and capture long-range dependent features of image detail regions. S64. Input the output feature map of the dilated convolution module into the multi-scale reconstruction branch. The multi-scale reconstruction branch includes multiple parallel sub-branches. Each sub-branch uses a different downsampling ratio and deconvolution operation to extract and restore structural and texture information at different resolution levels. S65. Within each sub-branch, the output of each stage is enhanced through a residual connection mechanism to preserve high-frequency details in the original structure and reduce information loss during the reconstruction process. S66. Perform feature fusion processing on the output of all branches, and integrate the multi-scale feature maps with dimension matching and contextual information through the convolutional fusion layer to generate a unified enhanced image feature representation. S67. The fused feature map is input into the final reconstruction decoding module, and the image is restored to its original size through continuous upsampling and nonlinear activation operations, forming a high-resolution image output with enhanced clarity. S68. Output the enhanced and reconstructed microscopic image, which has the characteristics of distinct texture layers, clear and sharp edges, and complete structural restoration, as input to the image quality assessment module.

[0014] Optionally, S7 specifically includes: S71. The enhanced image output by the residual hierarchical reconstruction network is used as the input image and sent to the image quality assessment module to initialize the image assessment process. S72. Align the enhanced image with the corresponding original input image or reference image, and crop both to a common effective area to ensure that subsequent evaluation metrics are calculated within a consistent spatial range; S73. In the structural similarity evaluation submodule, the image is divided into regions according to the preset block window. The local response relationship between the reference image and the enhanced image in the three dimensions of brightness, contrast and structure is calculated respectively. A structural similarity index heatmap is generated in combination. The global mean of the structural similarity index heatmap is summarized to obtain the image structural similarity index value. S74. In the image signal-to-noise ratio evaluation submodule, the error signal intensity between images is calculated by statistically analyzing the pixel-level difference. Combined with the pixel distribution intensity of the original image, a peak signal-to-noise ratio index is generated to reflect the overall information fidelity after image reconstruction. S75. In the edge preservation rate evaluation submodule, the edge operator is used to perform gradient detection operation on the image, extract the edge response maps of the enhanced image and the original image respectively, and perform matching statistics on the edge pixels in the overlapping area to calculate the edge energy preservation rate as the boundary sharpness evaluation result. S76. Combine the above three indicators of structural similarity, peak signal-to-noise ratio and edge preservation rate to construct an image quality evaluation vector, and use it as a quantitative expression of the current image stitching and reconstruction effect; S77. Compare the evaluation results of each group of images with the preset quality threshold. If any indicator fails to meet the standard, record the corresponding image number and output a quality warning signal for model error feedback. S78. Perform piece-by-piece quality inspection and evaluation on all image stitching blocks, and finally output a global quality score report for the entire image, which serves as the basis for generating system performance indicators and can be called by subsequent visualization or annotation modules.

[0015] Optionally, S8 specifically includes: S81. Construct a training sample image set containing various types of anomalies, wherein the distortion scenarios of the training sample images include seam position offset, texture blurring and degradation, local exposure difference and non-homogeneous background interference. S82. Perform pre-labeling processing on the collected training sample images, record the ideal registration position, real stitching boundary and corresponding reconstructed target image of each image, as a reference for subsequent supervision signals; S83. Input the training image into the structure-aware feature extraction network, the boundary attention stitching network, and the residual hierarchical reconstruction network, and complete the entire process of image registration, stitching, and enhancement according to the actual system process to generate the predicted output image. S84. In the training loss function design stage, define the image reconstruction error term and use the absolute difference between the predicted image and the target image in the pixel intensity dimension as the image reconstruction L1 loss. S85. Construct a boundary continuity loss term to detect structural breakpoints, gradient abrupt changes, and texture interruptions in the splicing region, and measure its consistency with the target boundary structure to constrain the splicing smoothness. S86. Construct a perceptual loss term. After converting the image to a high-dimensional feature space through a perceptual encoder, calculate the deep semantic feature differences to evaluate the perceptual quality of the overall structure and texture restoration of the image. S87. Construct an affine registration error penalty term. By measuring the difference between the predicted affine transformation parameters and the true parameters, constrain the geometric alignment accuracy of the image and improve the robustness of the model registration. S88. The image reconstruction L1 loss, boundary continuity loss, perceptual loss and affine registration error penalty term are jointly weighted to form a multi-objective optimization function, and end-to-end backpropagation training is performed to complete the joint optimization process of each sub-module, so as to improve the adaptability and image reconstruction accuracy under different deformation, brightness interference and structural blur conditions.

[0016] The beneficial effects of this invention are: This invention provides a method for seamless stitching and enhanced reconstruction of microscopic images based on deep learning. It addresses the technical problems of obvious image stitching misalignment, broken boundary structures, weak detail reconstruction capabilities, and lack of overall modeling in existing technologies. The method has systematically optimized and integrated the algorithm design and model architecture, achieving significant technical progress.

[0017] First, by constructing a structure-aware feature extraction network that includes a deformable convolutional coding module and a channel selection mechanism, this invention can effectively perceive non-rigid structural deformation and local tissue texture changes in microscopic images, significantly improving the ability of the feature extraction stage to express complex edges and morphologically diverse regions, and providing a high-quality input foundation for subsequent registration and fusion.

[0018] Secondly, by introducing an image registration module that includes a local response locator, an affine parameter estimator, and a bilinear interpolation reconstructor, an end-to-end learnable image geometric alignment mechanism is realized. This mechanism does not rely on traditional manual feature point matching and can maintain the structural consistency and spatial continuity between images even when there is uneven illumination, tissue misalignment, or blurring and degradation in the overlapping areas of the images.

[0019] Furthermore, this invention designs a boundary attention-driven stitching network, which utilizes salient boundary detection and region weight guidance mechanisms to dynamically adjust stitching weights during image fusion. Combined with a residual structure completion module, it repairs image edge breaks, significantly eliminating common problems in traditional stitching methods such as abrupt seams, structural inconsistencies, and texture breaks, achieving a natural and smooth image transition effect. The residual hierarchical reconstruction network, built based on dilated convolution stacking and multi-scale reconstruction branches, can introduce a large receptive field while maintaining image resolution, improving the clarity and edge sharpness of image details and effectively enhancing the local hierarchical structure and global texture information of microscopic images.

[0020] Finally, by constructing an image quality assessment mechanism that includes structural similarity index, peak signal-to-noise ratio and edge preservation rate, and combining multiple joint loss functions for end-to-end optimization during the training phase, this invention not only achieves full-process collaborative training, but also significantly improves the model's generalization ability and reconstruction accuracy under complex degradation conditions, adapting to diverse microscopic image scenarios.

[0021] In summary, this invention has achieved comprehensive improvements in image registration accuracy, stitching continuity, boundary fusion quality, and reconstruction clarity, overcoming key challenges in traditional technologies such as structural distortion, abrupt seams, and information loss. It has good practical value and promising prospects for promotion. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0023] Figure 1 This is an overall flowchart of a deep learning-based seamless stitching and enhancement reconstruction method for microscopic images proposed in this invention. Figure 2 This is a schematic diagram of the module structure of the structure-aware feature extraction network proposed in this invention; Figure 3 This is a flowchart illustrating the branch structure and enhancement process of the residual hierarchical reconstruction network proposed in this invention. Detailed Implementation

[0024] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0025] refer to Figure 1-3 A method for seamless stitching and enhanced reconstruction of microscopic images based on deep learning includes the following steps: S1. Acquire multiple original images with spatially overlapping areas obtained by a microscopic imaging device, and record the spatial location information and imaging parameters of each image; S2. Preprocess the original image to obtain a standardized image sequence; S3. Input the standardized image sequence into a structure-aware feature extraction network, which includes a deformable convolutional coding module and a channel selection module; S4. Input the feature map and the original image into the image registration module. The image registration module includes a local response locator, an affine parameter estimator, and a bilinear interpolation reconstructor. S5. Input the registered image sequence into a boundary attention-driven stitching network, which includes a boundary saliency detection module, a region weight guidance module, and a continuous structure completion module, and output a seamless image; S6. Input the seamless image into the residual hierarchical reconstruction network, which includes multiple dilated convolution stacked modules and multi-scale reconstruction branches, and outputs an enhanced reconstructed image. S7. Perform quality assessment on the enhanced reconstructed image, including calculating the structural similarity index, peak signal-to-noise ratio, and edge preservation rate; S8. Construct a training sample image set and perform end-to-end joint training using a joint loss optimization mechanism.

[0026] This invention provides an end-to-end optimized seamless stitching and enhanced reconstruction method for microscopic images. It integrates structure-aware feature extraction, spatial transformation registration, boundary attention stitching, and residual reconstruction network modules, establishing a closed-loop processing system from image acquisition to quality assessment. This method effectively solves the problems of structural misalignment, abrupt boundaries, and local blurring in traditional image stitching, significantly improving the spatial consistency, texture continuity, and clarity of the stitched images. It is particularly suitable for microscopic scenes with complex tissue morphology and unstable imaging quality, and possesses strong model adaptability and applicability.

[0027] In this embodiment, the preprocessing includes brightness normalization, edge-preserving filtering, gamma correction, and image color consistency adjustment.

[0028] By performing standardization operations such as brightness normalization, edge-preserving filtering, gamma correction, and color consistency adjustment before the image enters the neural network, interference caused by uneven exposure, tissue staining differences, or noisy backgrounds in microscopic images is effectively suppressed. This preprocessing scheme not only improves the statistical consistency and contrast stability of image data, but also builds a uniformly distributed data input foundation for subsequent deep feature learning, significantly improving the model's robustness and feature separability under non-ideal imaging conditions.

[0029] In this embodiment, S3 specifically includes: S31. Input the standardized image sequence into the input end of the structure-aware feature extraction network and map it to the initial feature tensor space to preserve the original spatial arrangement structure of the image. S32. In the first encoding stage of the structure-aware feature extraction network, a regular convolutional layer with a fixed receptive field is used to downsample the input image and perform batch normalization and activation function processing to obtain the basic low-level texture response map. S33. Input the low-level texture response map into the deformable convolutional coding module. The deformable convolutional coding module includes multiple deformable convolutional units. Each deformable convolutional unit introduces a learnable offset parameter to adjust the position of the standard sampling grid, thereby achieving adaptive modeling of the curved, non-rigid deformation and skewed areas of the target edge. S34. Concatenate the feature maps output by multiple deformable convolutional layers to form a composite texture encoding map containing different perceptual scales and response directions, and maintain the original spatial information without loss through residual connections. S35. The above composite texture encoding map is sent to the channel selection module. The channel selection module constructs multiple parallel convolution paths. Each convolution path uses a convolution kernel of a different size to form a multi-scale response structure for different receptive field scales. S36. In the channel selection module, a global average pooling operation is performed on the output of each path to calculate the response statistics of each channel. A channel attention mechanism is introduced to generate channel weighting coefficients. Based on the weighting result, the multi-path convolution outputs are fused to obtain a mid-to-high-level semantic feature map with the ability to adapt to differences in tissue density. S37. Input the fused mid-to-high-level feature map into the multi-branch stacking module. The multi-branch stacking module is composed of multiple deformable convolutional and channel selection units cascaded together. It enhances the encoding ability of complex texture transitions, boundary structures and micro-deformation regions by layer-by-layer enhancement. S38. Perform spatial alignment, feature normalization, and dimension fusion on the final output of all stacked units to output a high-dimensional feature tensor in a unified format. This feature tensor retains both the local texture details and global geometric structure information of the image, which is then used as input for the image registration module.

[0030] The proposed structure-aware feature extraction network integrates deformable convolutional modules and a channel selection mechanism, enabling adaptive modeling of complex geometries, non-rigid deformations, and local tissue density differences. Deformable convolution introduces a receptive field structure that learns offsets, effectively capturing regional features that are difficult to identify with conventional convolution, such as edge curvature and structural skew. The channel selection module enhances the perception of structures at different tissue scales through multi-scale path parallel convolution and attention-weighted fusion. The feature tensors output by this network possess both texture detail and geometric structure descriptive power, significantly improving the semantic expressiveness of images and providing high-quality support for subsequent registration.

[0031] In this embodiment, S4 specifically includes: S41. The high-dimensional feature map output by the structure-aware feature extraction network is concatenated and fused with the corresponding original image in the channel dimension to form an input tensor, which is then input into the image registration module. S42. In the image registration module, the local response locator scans the local region of the fused image to extract high response features in the overlapping region of the image, including significant edges, texture abrupt changes and alignment-sensitive regions, and outputs a local response heatmap. S43. Input the local response heatmap into the affine parameter estimator, which consists of a convolutional layer and a fully connected layer. Extract the geometric transformation pattern from the local response and output six affine transformation parameters, including horizontal displacement, vertical displacement, rotation angle, horizontal and vertical scaling factors and shearing factor. S44. Generate the corresponding two-dimensional affine transformation matrix according to the affine parameters, and construct the transformation mapping field in the target coordinate space; S45. Input the original image and the affine transformation matrix into a bilinear interpolation reconstructor. The bilinear interpolation reconstructor performs continuous sampling on the mapped coordinates and remaps the pixels of the original image according to the bilinear interpolation rules to generate an aligned image. S46. Apply the above registration operation to each pair of adjacent images, and repeat the local response extraction, affine parameter estimation and interpolation reconstruction process to output the registered image sequence. S47. Perform structural alignment detection on each overlapping region in the output image sequence, including contour closure, edge consistency and pixel drift error statistics. S48. If the structure alignment detection does not meet the preset threshold condition, the current image pair is re-input into the affine parameter estimator to update the transformation parameters, and the interpolation reconstruction operation is repeated until the condition is met, and finally a spatially consistent registered image sequence is formed.

[0032] This invention introduces a registration module that possesses strong robustness and high alignment accuracy, enabling accurate modeling of geometric relationships between microscopic images under unsupervised conditions. A local response locator identifies key structures in overlapping regions, an affine parameter estimator learns translation, rotation, scaling, and shearing relationships between images, and a bilinear interpolation reconstructor achieves continuous spatial remapping. Compared to traditional registration methods based on feature point matching or similarity metrics, this module exhibits differentiability and end-to-end optimization capabilities, significantly improving image stitching consistency and spatial alignment accuracy, especially demonstrating greater stability in cases of blurred tissue boundaries and severe deformation.

[0033] In this embodiment, S5 specifically includes: S51. Input the registered image sequence output by the image registration module into the boundary attention-driven stitching network in spatial order to initialize the image stitching process. S52. The response analysis of the overlapping area of ​​the image is performed by the boundary saliency detection module. The potential splicing seam area is extracted by the method based on local gradient statistics and texture difference mapping, and a saliency boundary heatmap is generated. S53. The salient boundary heatmap and the feature map of the original image are fused and input into the region weight guidance module. The region weight guidance module constructs a fusion priority map and calculates the image fusion weight based on the brightness balance, texture integrity and boundary continuity between the overlapping images. S54. Inside the region weight guidance module, different fusion weights are assigned to local regions of multiple overlapping images through an attention mechanism to form a weighted fusion image candidate set, and conflict regions are masked in the fusion image while retaining stable boundary regions. S55. The candidate image fusion result is sent to the continuous structure completion module to perform boundary repair operation. The continuous structure completion module is composed of multiple residual convolution units to perform contextual texture compensation and structural symmetry restoration on the structural interruption area at the splicing boundary. S56. In the continuous structure completion module, the feature information of adjacent non-seamless regions is diffused to the splicing boundary along the spatial gradient direction through the texture propagation mechanism to fill the boundary discontinuities and generate a fused image with continuous structure. S57. Perform a spatial consistency check on the stitched image to determine whether the boundary transition area meets the conditions of brightness consistency, contour closure and texture smoothness. S58. Output the seamless image after the above boundary detection, region weighting, and crack repair processing. The stitched image has smooth boundaries, continuous tissue structure, and complete texture details, and has a seamless stitching effect. It serves as the input for the subsequent image enhancement and reconstruction network.

[0034] The constructed boundary attention-driven stitching network, through the collaborative efforts of multiple modules including saliency detection, region guidance, and break completion, is specifically designed to model and optimize common problems in microscopic image stitching such as abrupt seams, edge jumps, and texture interruptions. This network not only dynamically identifies high-risk seam regions but also automatically adjusts image stitching weights based on fusion priority. Furthermore, it incorporates texture propagation and residual repair mechanisms to fill in the contextual structure of boundary breaks. Overall, this improves the structural smoothness, organizational continuity, and perceptual naturalness of the stitched images, avoiding obvious stitching artifacts produced by traditional image fusion methods.

[0035] In this embodiment, S6 specifically includes: S61. The seamless image output by the stitching network is fed into the input end of the residual hierarchical reconstruction network as the input image to initialize the image enhancement and reconstruction process. S62. In the initial module of the reconstruction network, standard convolutional coding operations are performed to extract shallow features of the image and perform normalization processing, providing a structural basis for subsequent multilayer perception. S63. Input shallow features into the dilated convolution stacking module, which consists of multiple dilated convolution layers with different dilation rates, to expand the receptive field while maintaining the resolution and capture long-range dependent features of image detail regions. S64. Input the output feature map of the dilated convolution module into the multi-scale reconstruction branch. The multi-scale reconstruction branch includes multiple parallel sub-branches. Each sub-branch uses a different downsampling ratio and deconvolution operation to extract and restore structural and texture information at different resolution levels. S65. Within each sub-branch, the output of each stage is enhanced through a residual connection mechanism to preserve high-frequency details in the original structure and reduce information loss during the reconstruction process. S66. Perform feature fusion processing on the output of all branches, and integrate the multi-scale feature maps with dimension matching and contextual information through the convolutional fusion layer to generate a unified enhanced image feature representation. S67. The fused feature map is input into the final reconstruction decoding module, and the image is restored to its original size through continuous upsampling and nonlinear activation operations, forming a high-resolution image output with enhanced clarity. S68. Output the enhanced and reconstructed microscopic image, which has the characteristics of distinct texture layers, clear and sharp edges, and complete structural restoration, as input to the image quality assessment module.

[0036] A residual hierarchical reconstruction network is employed to jointly enhance global and local details of images. A dilated convolution stacking module expands the receptive field to perceive long-range dependencies in the image, and a multi-scale branching structure supports parallel modeling of texture structures at different resolutions. Simultaneously, a residual mechanism is introduced to mitigate high-frequency detail loss, effectively enhancing image sharpness and structural layering capabilities. The reconstructed image output by this module exhibits sharp edges and distinct layers, capable of recovering blurred areas of detail caused by focusing deviations or unstable lighting during microscopic image acquisition, significantly improving the quality of image visual analysis.

[0037] In this embodiment, S7 specifically includes: S71. The enhanced image output by the residual hierarchical reconstruction network is used as the input image and sent to the image quality assessment module to initialize the image assessment process. S72. Align the enhanced image with the corresponding original input image or reference image, and crop both to a common effective area to ensure that subsequent evaluation metrics are calculated within a consistent spatial range; S73. In the structural similarity evaluation submodule, the image is divided into regions according to the preset block window. The local response relationship between the reference image and the enhanced image in the three dimensions of brightness, contrast and structure is calculated respectively. A structural similarity index heatmap is generated in combination. The global mean of the structural similarity index heatmap is summarized to obtain the image structural similarity index value. S74. In the image signal-to-noise ratio evaluation submodule, the error signal intensity between images is calculated by statistically analyzing the pixel-level difference. Combined with the pixel distribution intensity of the original image, a peak signal-to-noise ratio index is generated to reflect the overall information fidelity after image reconstruction. S75. In the edge preservation rate evaluation submodule, the edge operator is used to perform gradient detection operation on the image, extract the edge response maps of the enhanced image and the original image respectively, and perform matching statistics on the edge pixels in the overlapping area to calculate the edge energy preservation rate as the boundary sharpness evaluation result. S76. Combine the above three indicators of structural similarity, peak signal-to-noise ratio and edge preservation rate to construct an image quality evaluation vector, and use it as a quantitative expression of the current image stitching and reconstruction effect; S77. Compare the evaluation results of each group of images with the preset quality threshold. If any indicator fails to meet the standard, record the corresponding image number and output a quality warning signal for model error feedback. S78. Perform piece-by-piece quality inspection and evaluation on all image stitching blocks, and finally output a global quality score report for the entire image, which serves as the basis for generating system performance indicators and can be called by subsequent visualization or annotation modules.

[0038] The constructed image quality evaluation module comprehensively reflects the reconstruction effect of stitched images in terms of structural alignment, texture restoration, and boundary sharpness by jointly evaluating three metrics: structural similarity index, peak signal-to-noise ratio, and edge preservation rate. This evaluation mechanism not only supports quantitative evaluation of system performance but can also serve as a feedback signal for model training, enabling adaptive optimization. It is particularly suitable for automatic verification and output control in high-reliability image stitching systems.

[0039] In this embodiment, S8 specifically includes: S81. Construct a training sample image set containing various types of anomalies, wherein the distortion scenarios of the training sample images include seam position offset, texture blurring and degradation, local exposure difference and non-homogeneous background interference. S82. Perform pre-labeling processing on the collected training sample images, record the ideal registration position, real stitching boundary and corresponding reconstructed target image of each image, as a reference for subsequent supervision signals; S83. Input the training image into the structure-aware feature extraction network, the boundary attention stitching network, and the residual hierarchical reconstruction network, and complete the entire process of image registration, stitching, and enhancement according to the actual system process to generate the predicted output image. S84. In the training loss function design stage, define the image reconstruction error term and use the absolute difference between the predicted image and the target image in the pixel intensity dimension as the image reconstruction L1 loss. S85. Construct a boundary continuity loss term to detect structural breakpoints, gradient abrupt changes, and texture interruptions in the splicing region, and measure its consistency with the target boundary structure to constrain the splicing smoothness. S86. Construct a perceptual loss term. After converting the image to a high-dimensional feature space through a perceptual encoder, calculate the deep semantic feature differences to evaluate the perceptual quality of the overall structure and texture restoration of the image. S87. Construct an affine registration error penalty term. By measuring the difference between the predicted affine transformation parameters and the true parameters, constrain the geometric alignment accuracy of the image and improve the robustness of the model registration. S88. The image reconstruction L1 loss, boundary continuity loss, perceptual loss and affine registration error penalty term are jointly weighted to form a multi-objective optimization function, and end-to-end backpropagation training is performed to complete the joint optimization process of each sub-module, so as to improve the adaptability and image reconstruction accuracy under different deformation, brightness interference and structural blur conditions.

[0040] By constructing a multi-scene training dataset that includes seam misalignment, blur degradation, and exposure differences, and introducing multi-dimensional loss functions such as image reconstruction loss, boundary continuity loss, perceptual difference loss, and registration error penalty term, an end-to-end joint training system is formed, enabling multiple sub-modules to collaboratively optimize under a unified objective. This training mechanism significantly improves the model's adaptability, robustness, and generalization ability in complex microscopic image scenes, effectively avoiding the drawbacks of traditional single-loss training methods that are prone to getting trapped in local optima or overfitting.

[0041] Example 1: To verify the feasibility of this invention in practice, it was applied to a digital pathological slide-assisted diagnostic system. The acquisition of high-resolution microscopic images often requires digital modeling of the entire pathological slide through slide-by-slide scanning. During this process, due to the limited field of view of the imaging equipment, a complete field of view must be constructed by stitching together multiple local images. However, limited by factors such as uneven tissue slide thickness, differences in staining, fluctuations in microscope focal length, and equipment movement errors, the original image sequence often exhibits obvious structural misalignment, texture breaks, and inconsistent exposure in the boundary areas. This results in abrupt seams, blurred boundaries, and image discontinuities in the stitched results, severely affecting the accuracy of pathological feature extraction and subsequent diagnostic judgment.

[0042] To address the aforementioned issues, this embodiment selected breast cancer tissue slide data provided by the pathology department of a tertiary hospital as the experimental subject. A Leica Aperio AT2 fully automated digital slide scanner was used to scan the complete slides at 40× magnification, resulting in 160 original images. All images had a uniform resolution of 4096×4096 pixels, with approximately 12%–18% spatial overlap between them. During the acquisition process, some images were intentionally acquired under different exposure settings to simulate common clinical imaging scenarios such as uneven staining depth and uneven brightness distribution.

[0043] Within the framework of the proposed method, brightness normalization, edge-preserving filtering, and color consistency adjustment are first performed on all images to construct a standardized image sequence, which is then input into a structure-aware feature extraction network. This network is trained on the PyTorch platform, using 400 sets of image samples to construct the training set. Each set contains the image patches required for complete stitching, seam region annotations, and the enhanced target image. During training, a batch size of 8, an initial learning rate of 0.0002, and the Adam optimizer are used, with 200 training epochs.

[0044] In the image feature extraction stage, the introduced deformable convolutional units and channel selection modules effectively capture the semantic features of curved edges and skewed structures, making the registration module more accurate in image boundary localization. After image alignment is achieved through a spatial transformer network, a boundary attention-driven stitching network completes the fusion process, automatically assigning fusion weights in the seam areas and using a residual structure completion mechanism to repair broken textures.

[0045] The final stitched image has a size of 20480×16384 pixels, with complete image structure edges and a natural, seamless stitched area. Quantitative calculations by the quality assessment module show that the average Structural Similarity Index (SSIM) of the stitched image is 0.9532, the Peak Signal-to-Noise Ratio (PSNR) is 38.72dB, and the edge preservation rate reaches 0.916. These represent significant improvements compared to the traditional SIFT+RANSAC stitching scheme before the introduction of this invention (SSIM = 0.8704, PSNR = 31.56dB, edge preservation rate = 0.774).

[0046] Furthermore, to verify the reconstruction network's ability to enhance blurred regions, 10 randomly selected image fragments with blurred edges were analyzed. After processing using the residual hierarchical reconstruction module, the texture details of these images were significantly restored, and the edge response intensity was improved by 23.6%. In the visual subjective rating, 85% of pathologists commented that the edges were "clear" and the "tissue integrity was better," further demonstrating the effectiveness and practical value of this method in real microscopic scenes.

[0047] This embodiment demonstrates that the proposed deep learning-based seamless stitching and enhanced reconstruction method for microscopic images can significantly improve the spatial consistency and detail representation of multi-image stitched images, exhibiting excellent robustness and adaptability. It is applicable to various high-resolution microscopic imaging scenarios, and has broad prospects for application, especially in fields with high image quality requirements such as pathological auxiliary diagnosis and biological tissue reconstruction.

[0048] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for seamless stitching and enhanced reconstruction of microscopic images based on deep learning, characterized in that, The method comprises the following steps: S1, collecting multiple original images with spatial overlapping areas acquired by a microscopic imaging device, and recording spatial position information and imaging parameters of each image; S2, preprocessing the original images to obtain a standardized image sequence; S3, inputting the standardized image sequence into a structure-aware feature extraction network, the structure-aware feature extraction network comprising a deformable convolution encoding module and a channel selection module; S4, inputting the feature map and the original image into an image registration module, the image registration module comprising a local response locator, an affine parameter estimator, and a bilinear interpolation reconstructor; S5, inputting the registered image sequence into a boundary attention driven stitching network, the stitching network comprising a boundary saliency detection module, a region weight guiding module, and a continuous structure completion module, and outputting a seamless image; S6, inputting the seamless image into a residual level reconstruction network, the residual level reconstruction network comprising multiple hollow convolution stacking modules and multi-scale reconstruction branches, and outputting an enhanced reconstruction image; S7, performing quality evaluation on the enhanced reconstruction image, including calculating structural similarity index, peak signal-to-noise ratio, and edge retention rate; S8, constructing a training sample image set, and performing end-to-end joint training by using a joint loss optimization mechanism.

2. The method of claim 1, wherein, The preprocessing comprises brightness normalization, edge preservation filtering, gamma correction, and image color consistency adjustment.

3. The method of claim 1, wherein, The S3 specifically comprises: S31, inputting the standardized image sequence into the input end of the structure-aware feature extraction network, and mapping it to an initial feature tensor space; S32, performing feature down-sampling on the input image by using a normal convolution layer with a fixed receptive field, and performing batch normalization and activation function processing to obtain a basic low-level texture response map; S33, inputting the low-level texture response map into the deformable convolution encoding module, the deformable convolution encoding module comprising multiple deformable convolution units, each deformable convolution unit introducing a learnable offset parameter to adjust the position of a standard sampling grid; S34, performing cascade processing on the feature maps output by the multiple deformable convolution layers to form a composite texture encoding map containing different perception scales and response directions, and keeping the original spatial information from being lost through residual connection; S35, inputting the composite texture encoding map into the channel selection module, the channel selection module constructing multiple parallel convolution paths, each convolution path using a convolution kernel of different size to form a multi-scale response structure for different receptive field scales; S36, performing global average pooling operation on the output of each path in the channel selection module, calculating the response statistics of each channel, and introducing a channel attention mechanism to generate a channel weighting coefficient, and fusing the multi-path convolution output according to the weighting result to obtain a mid-high layer semantic feature map with organization density difference adaptation capability; S37, inputting the fused mid-high layer feature map into a multi-branch stacking module, the multi-branch stacking module being composed of multiple deformable convolutions and channel selection units in cascade, and enhancing the encoding capability of complex texture transitions, boundary structures, and microscopic deformation regions layer by layer. S38, spatially align, normalize and dimensionally fuse the final outputs of all stacked units, and output a high-dimensional feature tensor in a unified format, which retains both local texture details and global geometric structure information of the image.

4. The method of claim 1, wherein, The S4 specifically includes: S41, the high-dimensional feature map output by the structure perception feature extraction network is fused with the corresponding original image in the channel dimension to form an input tensor, which is input into the image registration module; S42, in the image registration module, the fused image is scanned in the local area by a local response locator to extract high-response features in the image overlap area, including significant edges, texture mutations and alignment-sensitive areas, and a local response heat map is output; S43, the local response heat map is input into an affine parameter estimator composed of convolutional layers and fully connected layers, which extracts geometric transformation patterns from the local response and outputs six affine transformation parameters, including horizontal displacement, vertical displacement, rotation angle, vertical-horizontal scaling factor and shear factor; S44, a corresponding two-dimensional affine transformation matrix is generated according to the affine parameters, and a transformation mapping field in the target coordinate space is constructed; S45, the original image and the affine transformation matrix are jointly input into a bilinear interpolation reconstructor, which performs continuous sampling on the mapping coordinates, remaps the original image pixels according to the bilinear interpolation rule, and generates an aligned image; S46, the above registration operation is applied to each group of adjacent image pairs, and the local response extraction, affine parameter estimation and interpolation reconstruction processes are repeatedly executed to output a sequence of registered images; S47, structure alignment detection is performed on each overlap area in the output image sequence, including contour closure, edge consistency and pixel drift error statistics; S48, if the structure alignment detection does not satisfy the preset threshold condition, the current image pair is re-input into the affine parameter estimator for transformation parameter updating, and the interpolation reconstruction operation is repeatedly executed until the condition is satisfied, and finally a sequence of spatially consistent registered images is formed.

5. The method of claim 1, wherein, The S5 specifically includes: S51, the sequence of registered images output by the image registration module is input into a boundary attention-driven stitching network in spatial order to initialize the image stitching process; S52, the image overlap area is analyzed by a boundary saliency detection module, and a potential stitching seam area is extracted by a method based on local gradient statistics and texture difference mapping to generate a saliency boundary heat map; S53, the saliency boundary heat map and the feature map of the original image are fused and input into a region weight guiding module, which constructs a fusion priority map and calculates image fusion weights according to the brightness balance, texture integrity and boundary continuity between the overlapping images; S54, inside the region weight guiding module, different fusion weights are given to local areas of multiple overlapping images through an attention mechanism to form a weighted fusion image candidate set, and conflict areas are shielded in the fusion image while retaining stable boundary areas; S55, send the candidate image fusion result into a continuous structure completion module, perform boundary repair operation, the continuous structure completion module is composed of multilayer residual convolution unit, and the context texture compensation and structure symmetry restoration are carried out to the structure interruption area at the splicing boundary; S56, in the continuous structure completion module, the characteristic information of adjacent non-seam area is diffused along the spatial gradient direction to the splicing boundary through the texture propagation mechanism, the boundary breakpoint is filled, and the fusion image with structure continuity is generated; S57, the splicing image is subjected to spatial consistency check, and whether the boundary transition region satisfies the conditions of brightness consistency, contour closure and texture smoothness is judged; S58, output the seamless image after the boundary detection, region weighting and broken line repair processing, the splicing image boundary is smooth, the organization structure is continuous, and the texture details are kept complete.

6. The method of claim 1, wherein, The S6 specifically comprises: S61, the seamless image output by the splicing network is sent into the input end of the residual level reconstruction network as an input image, and the image enhancement reconstruction process is initialized; S62, in the initial module of the reconstruction network, standard convolution coding operation is performed, shallow features of the image are extracted and normalized; S63, the shallow features are input into the hollow convolution stacking module, the hollow convolution stacking module is composed of a plurality of hollow convolution layers with different expansion rates, the receptive field is expanded under the condition that the resolution is unchanged, and long-range dependence features of the image detail area are captured; S64, the output feature map of the hollow convolution module is input into the multi-scale reconstruction branch, the multi-scale reconstruction branch includes a plurality of parallel sub-branches, each sub-branch adopts different scale down-sampling ratio and deconvolution operation, and structure information and texture information of different resolution levels are extracted and restored; S65, in each sub-branch, each level output is enhanced through the residual connection mechanism, the high-frequency details in the original structure are retained, and the information loss in the reconstruction process is weakened; S66, the outputs of all branches are subjected to feature fusion processing, the multi-scale feature maps are subjected to dimension matching and context information integration through the convolution fusion layer, and a unified enhanced image feature representation is generated; S67, the fused feature map is input into the final reconstruction decoding module, and through continuous up-sampling and non-linear activation operation, the original image size is restored, and a high-resolution image output with enhanced clarity is formed; S68, output the microscopic image after the enhancement reconstruction processing, the microscopic image has the characteristics of clear texture level, clear and sharp edge and complete structure recovery, and serves as the input of the image quality evaluation module.

7. The method of claim 1, wherein, The S7 specifically comprises: S71, the enhanced image output by the residual level reconstruction network is sent into the image quality evaluation module as an input image, and the image evaluation process is initialized; S72, the enhanced image is aligned with the corresponding original input image or reference image, and both are cropped to a common effective area; S73, in the structural similarity evaluation submodule, the image is regionally divided according to a preset block window, the local response relationship of the reference image and the enhanced image in the three dimensions of brightness, contrast and structure is calculated respectively, a structural similarity index heat map is generated comprehensively, and the global mean of the structural similarity index heat map is summarized to obtain an image structural similarity index value; S74, in the image signal-to-noise ratio evaluation submodule, the error signal strength between images is calculated through pixel-level difference statistics, and a peak signal-to-noise ratio index is generated in combination with the pixel distribution intensity of the original image; S75, in the edge retention rate evaluation submodule, an edge operator is used to perform a gradient detection operation on the image, the edge response graphs of the enhanced image and the original image are extracted respectively, and the edge pixels in the overlapping area are matched and counted, and the edge energy retention rate is calculated as the boundary sharpness evaluation result; S76, the structural similarity, the peak signal-to-noise ratio and the edge retention rate are combined to construct an image quality evaluation vector, which is used as a quantitative expression of the current image stitching and reconstruction effect; S77, the evaluation results of each group of images are compared with a preset quality threshold, if any index does not meet the standard, the corresponding image number is recorded and a quality warning signal is output; S78, all image stitching blocks are detected and evaluated one by one, and finally a global quality score report of the whole image is output, which is used as the basis for generating system performance indicators.

8. The method of claim 1, wherein, The S8 specifically comprises: S81, a training sample image set containing multiple types of abnormalities is constructed, and the distortion scenarios of the training sample images include joint position offset, texture blur degradation, local exposure difference and non-homogeneous background interference; S82, the collected training sample images are pre-labeled, and the ideal registration position, the real stitching boundary and the corresponding reconstructed target image of each image are recorded; S83, the training images are input into the structure perception feature extraction network, the boundary attention splicing network and the residual hierarchical reconstruction network, and the whole process of image registration, stitching and enhancement is completed according to the actual system process to generate a predicted output image; S84, in the training loss function design stage, an image reconstruction error term is defined, and the absolute difference value of the predicted image and the target image in the pixel intensity dimension is used as the image reconstruction L1 loss; S85, a boundary continuity loss term is constructed, the structure breakpoint, gradient mutation and texture interruption of the stitching area are detected, and the consistency between them and the target boundary structure is measured to constrain the stitching smoothness; S86, a perception loss term is constructed, the image is converted to a high-dimensional feature space through a perception encoder, and the difference of deep semantic features is calculated; S87, an affine registration error penalty term is constructed, the difference between the predicted affine transformation parameters and the real parameters is measured to constrain the geometric alignment accuracy of the image, and the model registration robustness is improved; S88, the image reconstruction L1 loss, the boundary continuity loss, the perception loss and the affine registration error penalty term are jointly weighted to form a multi-objective optimization function, and end-to-end back propagation training is performed to complete the joint optimization process of each submodule.

Images