Microscope time sequence scanning image splicing method based on global position optimization
By employing global position optimization and temporal alignment methods, the problems of cumulative error and temporal jitter in microscopic temporal scanning image stitching are solved, achieving high-precision, artifact-free image stitching suitable for biomedical microscopic imaging and tissue section analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-24
- Publication Date
- 2026-04-03
AI Technical Summary
Existing microscopic time-series scanning image stitching suffers from cumulative errors, time jitter artifacts, and local optima, resulting in misalignment, deformation, and artifacts in the stitched images, which is particularly unsatisfactory when processing large-scale scenes or dynamic samples.
A two-stage approach based on global position optimization is adopted. First, the relative displacement between images is calculated through phase correlation. Then, the position of all images is adjusted using a global position optimization algorithm. Finally, temporal alignment is performed through feature point matching to ensure that the images are in the global optimal position and time consistent.
It achieves high-precision, artifact-free image stitching, eliminates accumulated errors and timing jitter, and improves stitching accuracy and overall consistency, making it particularly suitable for biomedical microscopic imaging and tissue section analysis.
Smart Images
Figure CN121788337A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of microscope image processing and stitching, specifically to a microscope temporal scanning image stitching method based on global position optimization, which is applied to scenarios such as biomedical microscopic imaging and tissue section analysis. Background Technology
[0002] Currently, time-series microscopy scanning often employs traditional sequential stitching algorithms. This involves first registering the current frame pairwise with the previous or neighboring frames, and then stacking them sequentially to generate a larger stitched image. This method has the following main drawbacks when processing large scenes or long image sequences:
[0003] 1. Cumulative error: Due to the sequential registration from front to back, the small deviations in each registration will gradually accumulate in subsequent steps, resulting in obvious misalignment or deformation at both ends of the final stitched image.
[0004] 2. Temporal jitter artifacts: For dynamic samples that change over time (such as slight movement of cell communities), sequential stitching cannot guarantee the consistency of each frame relative to the global reference, which can easily introduce temporal jitter and produce artifacts in subsequent dynamic analysis (such as motion tracking and deformation assessment).
[0005] 3. Local Optimum: Sequential registration often adopts a local optimum strategy, which only considers the overlapping areas of adjacent images. It lacks global constraints and cannot correct the overall misalignment under the combined effect of multiple images.
[0006] Application content
[0007] Addressing the shortcomings of traditional sequential stitching algorithms commonly used in current microscopy temporal scanning, this application aims to provide a microscopy image stitching scheme that can simultaneously eliminate accumulated errors and temporal jitter, and optimize the position of each frame globally. The specific technical solution and features of this application are as follows:
[0008] This application proposes a microscopic temporal scanning image stitching method based on global position optimization, which involves two-stage optimization:
[0009] 1. Spatial Dimension Optimization: The relative displacement of all images in the first frame and their neighboring images is calculated by using the phase correlation method. Then, the global position optimization algorithm is used to adjust the position of all images to obtain the global position parameters with the smallest error.
[0010] 2. Temporal dimension optimization: By using a feature point-based temporal alignment method, the images at each time step are aligned to the global optimal position of the first frame, thereby minimizing temporal jitter.
[0011] The specific steps are as follows:
[0012] 1. Microscope scanning images: Control the microscope to move the stage and take images in a specific order.
[0013] 2. Calculate the relative displacement between adjacent images:
[0014] S1. Perform frequency domain transformation on the two input images I1 and I2 to be registered to obtain G1 and G2.
[0015] S2. Calculate the phase correlation result R:
[0016]
[0017] Among them, G * Let G be the conjugate of G, and |·| denote the modulo operation.
[0018] S3. Perform an inverse Fourier transform on R to obtain the cross-correlation image r. The position of the maximum value of the cross-correlation image is the translation amount (dx, dy) between the images.
[0019] r(x,y)=F -1 R
[0020] 3. Global positioning optimization:
[0021] S1. Initialize the global position p of all images. i =(x i ,y i ), where i is the image index.
[0022] S2. Construct the loss function L:
[0023]
[0024] Where E is the set of adjacent image pairs, w ij t represents the confidence weight of the relative displacement. ij Let be the relative displacement between images i and j.
[0025] S3. Use the Adam optimizer for iterative optimization and update the global position parameters.
[0026]
[0027] Where η is the learning rate. This is the gradient of the loss function.
[0028] S4. Stop iterating when the loss function L converges.
[0029] 4. Stitch together a large image based on the global position: Based on the optimized global position p i All images are stitched together to form the first large-size image.
[0030] 5. Align all images to the first frame:
[0031] S1, For images I at subsequent time points t Feature point detection is performed to obtain the key point set K. t .
[0032] S2, Matching key point K t Combine the keypoint K1 of the first frame to obtain the set of matching pairs M.
[0033] S3, Calculate the time-series displacement (dx) t ,dy t ):
[0034]
[0035] Here, f(k) and f(m) are the feature vectors of key points k and m.
[0036] 6. A large time-series image is obtained by stitching and fusion: A linear fusion method is used to perform a weighted average of the pixel values in the overlapping areas.
[0037]
[0038] Here, α is a weighting coefficient, which is usually adjusted according to the tissue distribution of the overlapping areas of the image.
[0039] Preferably, the microscope scans the image in a left-to-right and top-to-bottom order, and the scanned image size is 2048×2048 pixels with a resolution of 0.1μm / pixel.
[0040] Preferably, in the global position optimization, the confidence weight w of the relative displacement of the loss function L is... ij Set it to 0.95.
[0041] Preferably, in the global position optimization, the learning rate η of the Adam optimizer is set to 0.1, and the maximum number of iterations is 200.
[0042] Preferably, in the image alignment process, the feature point detection performed on the images at subsequent time points involves approximately 500 key points and approximately 300 matching pairs.
[0043] Preferably, in the linear fusion method, the weighting coefficient α = 0.6, and the final time-series stitched large image size is 8192×8192 pixels, with stitching accuracy reaching the sub-pixel level.
[0044] In summary, this application addresses the shortcomings of traditional sequential stitching algorithms commonly used in current microscopy temporal scanning in three aspects: accumulated error, temporal jitter artifacts, and local optima. By introducing global position optimization and temporal alignment methods, it processes the image from both spatial and temporal dimensions. Global position optimization solves the problems of accumulated error and local optima, while the temporal alignment method eliminates temporal jitter artifacts, thus overcoming the deficiencies of existing technologies and achieving high-precision, artifact-free microscopy image stitching. The specific beneficial effects are as follows:
[0045] 1. Eliminate cumulative errors: Through global position optimization, the problem of error accumulation caused by sequential registration is avoided, thus improving the stitching accuracy.
[0046] 2. Minimize temporal jitter: Use feature point matching for temporal alignment to ensure consistency of each frame in the time dimension and reduce artifacts in dynamic analysis.
[0047] 3. Global optimization: It takes into account the interaction of all images and achieves the best overall stitching effect. Attached Figure Description
[0048] Figure 1 This is a schematic diagram of the overall process of this application;
[0049] Figure 2 This is a schematic diagram of the global optimization process for this application. Detailed Implementation
[0050] The term "embodiment" used herein, as an example, is not necessarily to be construed as superior to or better than other embodiments. Performance testing in the embodiments of this application, unless otherwise specified, employs conventional testing methods in the art. It should be understood that the terminology used in this application is merely for describing particular implementations and is not intended to limit the scope of this disclosure.
[0051] Unless otherwise stated, the technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; other experimental methods and technical means not specifically mentioned herein refer to experimental methods and technical means commonly used by one of ordinary skill in the art.
[0052] To better illustrate the content of this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented even without certain specific details. In the embodiments, some methods, means, instruments, and devices well-known to those skilled in the art are not described in detail in order to highlight the main points of this application.
[0053] The specific embodiments of this application are described below to enable those skilled in the art to understand this application. However, it should be understood that this application is not limited to the scope of the specific embodiments. For those skilled in the art, as long as the various changes are within the spirit and scope of this application as defined and determined by the appended claims, these changes are obvious. All applications that utilize the concept of this application are protected.
[0054] Example
[0055] Please see Figure 1 and Figure 2 The flowchart illustrates an embodiment of the proposed global position optimization-based microscopic temporal scanning image stitching method. The specific implementation details are as follows:
[0056] Example 1
[0057] 1. Microscope scanning images: Control the microscope to move the stage and take a series of microscopic images in order from left to right and from top to bottom. The image size is 2048×2048 pixels and the resolution is 0.1μm / pixel.
[0058] 2. Calculate the relative displacement between adjacent images:
[0059] S1. Perform grayscale processing on each pair of adjacent images.
[0060] S2. Perform Fourier transform to obtain the frequency domain image.
[0061] S3. Calculate the phase correlation result to obtain the relative displacement (dx, dy). For example, in a pair of images, the calculated relative displacement is (12.3, -8.7) pixels.
[0062] 3. Global positioning optimization:
[0063] S1. Initialize the global position of all images to (0,0).
[0064] S2. Set the confidence weight w for the relative displacement. ij It is 0.95.
[0065] S3. Use the Adam optimizer with a learning rate η = 0.1 and a maximum number of iterations of 200.
[0066] The iterative optimization process is shown in the table below:
[0067]
[0068]
[0069] 4. Stitching large image: Based on the optimized global position, all images are stitched together into a first large image with a size of 8192×8192 pixels.
[0070] 5. Timing alignment:
[0071] S1. Feature point detection is performed on images at subsequent time points, and the number of key points detected is approximately 500.
[0072] S2. Match key points to obtain approximately 300 matching pairs.
[0073] S3. Calculate the temporal displacement. For example, the temporal displacement of an image at a certain moment is (5.2, 3.8) pixels.
[0074] 6. Image fusion:
[0075] S1. A linear fusion method is adopted, with a weighting coefficient α = 0.6.
[0076] S2. Perform a weighted average on the overlapping areas to eliminate the influence of seams.
[0077] S3. Finally, a large time-series stitched image is obtained, with a size of 8192×8192 pixels and a stitching accuracy of sub-pixel level.
[0078] Through the detailed implementation methods described above, the microscopic temporal scanning image stitching method based on global position optimization of this application processes images from both spatial and temporal dimensions. Global position optimization solves the problems of accumulated errors and local optima, while the temporal alignment method eliminates temporal jitter artifacts, thereby overcoming the shortcomings of existing technologies and achieving high-precision, artifact-free microscopic image stitching.
[0079] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
[0080] The technical solutions and technical details disclosed in the embodiments of this application are merely illustrative of the application concept and do not constitute a limitation on the technical solutions of this application. Any conventional changes, substitutions or combinations made to the technical details disclosed in the embodiments of this application have the same application concept as this application and are within the protection scope of the claims of this application.
Claims
1. A method for stitching microscopic temporal scanning images based on global position optimization, characterized in that, Includes the following steps: S1. Calculate the relative displacement of all images in the first frame registered with their neighboring images using the phase correlation method; S2. Perform global position optimization on all images in the spatial dimension to obtain the global position parameters with the smallest error; S3. Using a feature-point-based temporal alignment method, the images at each time step are aligned to the global optimal position of the first frame in the time dimension and then stitched and fused together.
2. The method according to claim 1, characterized in that, The phase correlation method includes the following steps: S1. Perform frequency domain transformation on the two input images I1 and I2 to be registered to obtain G1 and G2; S2, Calculate the relevant result R: Among them, G * Let G be the conjugate of G, and |·| denote the modulo operation. S3. Perform an inverse Fourier transform on R to obtain the cross-correlation image r. The position of the maximum value of the cross-correlation image is the translation amount (dx, dy) between the images.
3. The method according to claim 1, characterized in that, The global location optimization includes the following steps: S1. Initialize the global position p of all images. i =(x i ,y i ), where i is the image index; S2. Construct the loss function L: Where E is the set of adjacent image pairs, w ij t represents the confidence weight of the relative displacement. ij This represents the relative displacement between images i and j; S3. Use the Adam optimizer for iterative optimization and update the global position parameters. Where η is the learning rate. The gradient of the loss function; S4. Stop iterating when the loss function L converges.
4. The method according to claim 1, characterized in that, The feature-point-based temporal alignment method includes the following steps: S1, For images I at subsequent time points t Feature point detection is performed to obtain the key point set K. t . S2, Matching key point K t Combine the keypoint K1 of the first frame to obtain the set of matching pairs M. S3, Calculate the time-series displacement (dx) t ,dy t ): Here, f(k) and f(m) are the feature vectors of key points k and m.
5. The method according to claim 1, characterized in that, The stitching and fusion step employs a linear fusion method, which performs a weighted average of the pixel values in the overlapping areas: Here, α is a weighting coefficient, which is usually adjusted according to the tissue distribution of the overlapping areas of the image.
6. The method according to claim 1, characterized in that, The microscope moving stage takes images in a left-to-right, top-to-bottom order, with an image size of 2048×2048 pixels and a resolution of 0.1μm / pixel.
7. The method according to claim 1, characterized in that, In the global position optimization step, a confidence weight w for the relative displacement is set. ij The value is 0.95, using the Adam optimizer, with a learning rate η = 0.1 and a maximum number of iterations of 200.
8. The method according to claim 1, characterized in that, In the temporal alignment method, feature point detection is performed on the images at subsequent time points, with 500 key points detected and 300 matching pairs.
9. The method according to claim 5, characterized in that, In the image stitching and fusion step, the weighting coefficient α = 0.6, and the final time-series stitched large image size is 8192×8192 pixels, with a stitching accuracy of sub-pixel level.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.