Three-dimensional biological image block splicing method and system based on global constraint optimization

Abstract

The application relates to the field of computer technology, in particular to a three-dimensional biological image block splicing method and system based on global constraint optimization, which comprises the following steps: acquiring a plurality of three-dimensional biological image blocks with initial spatial position information; constructing a splicing model according to the adjacent relationship among the three-dimensional biological image blocks, taking the spatial pose of each three-dimensional biological image block as an optimization variable; introducing a plurality of global constraint conditions in the splicing model, and uniformly modeling the overlapping relationship, vacancy condition and overall smoothness among the three-dimensional biological image blocks; jointly optimizing the global constraint conditions, solving the optimal spatial pose parameters of each three-dimensional biological image block, and completing the overall splicing reconstruction of the three-dimensional biological image according to the parameters, so that the three-dimensional biological image with consistent spatial structure and continuous morphology is obtained. Therefore, the high-precision reconstruction of the large-scale three-dimensional biological image is realized, and the continuity and consistency of the splicing result in the spatial structure are ensured.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a method and system for stitching together three-dimensional biological images based on global constraint optimization. Background Technology

[0002] With the increasing demands for precision and scale in the analysis of biological tissue structures in life science research, three-dimensional biological imaging technology has been widely applied in fields such as neuroscience, developmental biology, and pathological analysis. Due to the objective limitations of microscopic imaging systems in terms of field of view and resolution, large biological samples usually cannot obtain a complete three-dimensional structure through a single imaging session. Instead, a block imaging approach is used to acquire multiple local three-dimensional image data, which are then stitched together to reconstruct a complete three-dimensional image.

[0003] The core task of 3D biological image patch stitching is to determine the optimal positional relationship of each image patch in a unified spatial coordinate system, so that the stitched image maintains continuity and consistency in spatial structure. However, in practical applications, the spatial relationship between image patches is often uncertain due to factors such as sample preparation errors, imaging noise, sample deformation, and limited overlap between image patches, posing a significant challenge to high-precision stitching. Especially when there are a large number of image patches and a large stitching area, local errors can easily accumulate during the stitching process, thus affecting the overall reconstruction effect.

[0004] Therefore, how to ensure the overall consistency control of multiple 3D images while maintaining local registration accuracy has become an important research problem in the field of 3D biological image block stitching. Existing technical solutions for 3D biological image block stitching mainly fall into two categories: stitching methods based on local registration and stitching methods that incorporate global optimization concepts.

[0005] One type of method focuses on local image registration. It estimates the relative displacement or transformation relationship between adjacent or overlapping image blocks by performing feature matching or similarity calculation on adjacent or overlapping image blocks, and then stitches them together step by step in a predetermined order. This type of method is relatively simple to implement and can achieve certain results when the number of image blocks is small or the overlapping area is sufficient. However, it is essentially still a block-by-block stitching strategy and lacks a unified constraint on the overall structure.

[0006] Another type of method, based on local registration results, introduces a global optimization concept, unifying the spatial relationships of multiple image blocks into a global optimization problem. For example, by constructing an optimization model with image block poses as variables and inter-block relative relationships as constraints, the positions of all image blocks are jointly solved to reduce the cumulative impact of local errors in the overall stitching process. Compared to purely local stitching strategies, this type of method improves the overall consistency of the stitching results to a certain extent.

[0007] In addition, some studies have attempted to combine graph optimization or energy minimization frameworks to introduce factors such as overlap consistency and smoothness into the stitching model, in order to improve the stability and robustness of 3D image stitching. These methods provide new ideas for 3D image block stitching, but in complex biological image scenarios, there is still room for improvement in the unified modeling and collaborative optimization of multiple constraints.

[0008] In summary, 3D biological image patch stitching technology has been widely used in microscopic imaging data reconstruction for many years. Existing methods can achieve spatial registration between image patches and complete 3D structural reconstruction to a certain extent. However, in large-scale, complex 3D biological image scenarios, existing technologies still have the following shortcomings: 1. Existing stitching methods are mostly based on the local registration relationship of adjacent image blocks, and usually adopt a block-by-block or sequential stitching strategy. They lack a unified modeling mechanism for the spatial relationship of all image blocks, which can easily lead to the accumulation of registration errors when there are a large number of image blocks, resulting in overall pose drift and structural misalignment. 2. Some methods that introduce the idea of ​​global optimization usually only model and optimize for a single constraint, which makes it difficult to simultaneously take into account multiple stitching requirements such as image patch overlap control, gap suppression and overall structural smoothness, resulting in insufficient stability of the overall stitching result. 3. Existing technologies lack effective overall constraints on image block deformation during the stitching process, which can easily lead to local overstretching or compression, destroying the true structural morphology of biological tissues in three-dimensional space. 4. Under complex 3D biological image data conditions, existing stitching methods are quite sensitive to initial pose errors, often requiring manual intervention or multiple adjustments, which makes it difficult to meet the practical application requirements of efficient and automated stitching. Summary of the Invention

[0009] This invention provides a method and system for 3D biological image block stitching based on global constraint optimization, which realizes high-precision reconstruction of large-scale 3D biological images, thereby ensuring the continuity and consistency of the stitching results in spatial structure.

[0010] According to an embodiment of the present invention, a method for segmenting and stitching three-dimensional biological images based on global constraint optimization is provided, comprising the following steps: Acquire multiple 3D biological image patches with initial spatial location information; A stitching model is constructed based on the adjacency relationship between 3D biological image blocks, and the spatial pose of each 3D biological image block is used as an optimization variable. Multiple global constraints are introduced into the stitching model to uniformly model the overlap relationship, gap situation and overall smoothness between 3D biological image blocks; By jointly optimizing the global constraints, the optimal spatial pose parameters of each 3D biological image block are solved, and the overall stitching and reconstruction of the 3D biological image is completed accordingly, resulting in a 3D biological image with consistent spatial structure and continuous morphology.

[0011] Furthermore, a stitching model is constructed based on the adjacency relationships between the 3D biological image patches, and the spatial pose of each 3D biological image patch is used as an optimization variable, including: A stitching model is constructed by obtaining the adjacency relationship between image patches through overlap detection. Rigid registration based on maximizing mutual information and initial pose estimation using a preliminary registration algorithm.

[0012] Furthermore, constructing a stitching model by obtaining the adjacency relationships between image patches through overlap detection includes: All image blocks are abstracted as graph nodes, and the relationships between blocks are represented by edges, forming an undirected graph.

[0013] Furthermore, rigid registration based on maximizing mutual information and estimation of the initial pose using a preliminary registration algorithm includes: Calculate the transformation matrix for adjacent blocks to obtain the initial translation and rotation parameters.

[0014] Furthermore, various global constraints are introduced into the stitching model to uniformly model the overlap relationship, gaps, and overall smoothness between 3D biological image patches, including: Multiple global constraints are introduced to perform unified modeling of the splicing model, forming an energy function; The optimal pose is solved by jointly optimizing the global constraint model and using a least squares solver.

[0015] Furthermore, it also includes: Based on the optimization results, a legality verification is performed. If the verification passes, it is checked whether the stitching target has been achieved. If not, the model is updated, the constraint weights are adjusted, and a new model is built in a loop to continue optimization. If the verification fails or the target is achieved, the algorithm is terminated, and the image blocks are fused according to the optimal pose to output a complete three-dimensional biological image.

[0016] Furthermore, it also includes: A batch of training data is prepared using an image patch generation algorithm based on simulated deformation as the training set for the algorithm's optimization constraints.

[0017] Furthermore, noise blocks that deviate from the pose are introduced into the dataset, and the noise blocks are made to have energy terms pointing to the original structure. With training data that has structural consistency and appropriate noise data, the model training can achieve a global error correction function.

[0018] Furthermore, it also includes: The stitching is optimized starting from the initial pose, and all constraint energy values ​​are recorded. At each optimization step, the similarity of the current pose is checked for reasonableness, and back-optimization is used for verification. By applying the constraint model backward from the current pose, we can determine whether the energy of the backward optimization is reasonable compared with the error of the forward optimization, and ensure that the optimized pose always satisfies global consistency. The system determines the relative relationship between the current patch and its neighboring blocks in real time, and judges whether it is stable by whether the missing volume is minimized and whether the smoothness is consistent.

[0019] According to another embodiment of the present invention, a three-dimensional biological image block stitching system based on global constraint optimization is provided, comprising: The data input module is used to acquire multiple 3D biological image blocks with initial spatial location information; The model building module is used to construct a stitching model based on the adjacency relationship between 3D biological image patches, and the spatial pose of each 3D biological image patch is used as an optimization variable. The optimization solution module is used to introduce various global constraints into the stitching model and to uniformly model the overlap relationship, gap situation and overall smoothness between 3D biological image blocks; The output module is used to jointly optimize global constraints, solve for the optimal spatial pose parameters of each 3D biological image block, and complete the overall stitching and reconstruction of the 3D biological image to obtain a 3D biological image with consistent spatial structure and continuous morphology.

[0020] A storage medium storing a program file capable of implementing any of the above-mentioned three-dimensional biological image block stitching methods based on global constraint optimization.

[0021] A processor for running a program, wherein the program executes any of the above-mentioned three-dimensional biological image block stitching methods based on global constraint optimization.

[0022] The three-dimensional biological image block stitching method and system based on global constraint optimization in this invention model the three-dimensional biological image block stitching problem into a unified global optimization framework. It uses multiple constraints to jointly solve the spatial pose of image blocks, thereby achieving high-precision reconstruction of large-scale three-dimensional biological images and ensuring the continuity and consistency of the stitching results in spatial structure. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a flowchart of the overall solution. Figure 2 Example images for the training set; Figure 3 To optimize the model's principle block diagram; Figure 4 This is a schematic diagram of the constraint verification mechanism. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] In life science research, the stitching and reconstruction of three-dimensional biological images is a crucial step in achieving large-scale, high-resolution structural analysis. Due to the limitations of the field of view in microscopic imaging, large biological samples typically require the acquisition of multiple local three-dimensional image blocks through block imaging, followed by stitching to reconstruct the complete three-dimensional structure. However, in actual imaging and processing, existing three-dimensional image stitching methods still have significant shortcomings in terms of global consistency, structural continuity, and deformation control due to factors such as slice damage, imaging noise, non-rigid tissue deformation, and insufficient overlap between blocks.

[0027] Existing methods mostly rely on local feature matching or block-by-block registration strategies, lacking global constraints on the overall stitching result. This can easily lead to problems such as pose drift, structural misalignment, and local overlap or gaps during the accumulation and stitching of multiple images. Especially in large-scale 3D biological image scenarios, these errors will gradually amplify, seriously affecting the spatial accuracy and structural reliability of the final 3D reconstruction.

[0028] Therefore, how to introduce a unified global constraint mechanism in the process of 3D biological image block stitching, and optimize the spatial position and deformation of each image block as a whole, so as to improve the consistency and stability of the overall stitching while ensuring the accuracy of local registration, is a key technical problem that urgently needs to be solved in the current field of 3D biological image stitching.

[0029] To address the aforementioned shortcomings, the objective of this invention is: A method and system for stitching three-dimensional biological images based on global constraint optimization is proposed. The spatial pose of all image blocks is uniformly modeled and jointly solved to reduce the accumulation of local registration errors during the stitching process. Global constraints are introduced into the stitching model to collaboratively optimize the overlapping relationship, gaps, and overall smoothness between image blocks, so as to obtain a stitching result with consistent structure and reasonable space. By constraining and controlling the overall deformation of image blocks, unreasonable local deformations are suppressed, and the true morphological characteristics of biological tissues in three-dimensional space are maintained. To develop a stable stitching scheme suitable for large-scale 3D biological images, reducing the need for manual intervention and improving stitching efficiency and result reliability.

[0030] Example 1 This invention provides a method for stitching together three-dimensional biological images based on global constraint optimization. The technical solution includes: acquiring multiple three-dimensional biological image blocks with initial spatial position information; constructing a stitching model based on the adjacency relationship between the three-dimensional biological image blocks, using the spatial pose of each three-dimensional biological image block as an optimization variable; introducing various global constraints into the stitching model to uniformly model the overlap relationship, gaps, and overall smoothness between the three-dimensional biological image blocks; solving for the optimal spatial pose parameters of each three-dimensional biological image block by jointly optimizing the global constraints, and thereby completing the overall stitching and reconstruction of the three-dimensional biological image, thus obtaining a three-dimensional biological image result with consistent spatial structure and continuous morphology.

[0031] This invention provides a method for 3D biological image block stitching based on global constraint optimization. This method models the 3D biological image block stitching problem as a unified global optimization framework and uses multiple constraints to jointly solve the spatial pose of image blocks, thereby achieving high-precision reconstruction of large-scale 3D biological images and ensuring the continuity and consistency of the stitching results in spatial structure.

[0032] Specifically, this invention uses multiple local 3D biological image patches as input data, constructs a global stitching model using a graph optimization framework, treats the image patch pose as an optimization variable, introduces missing image suppression and smoothness constraints, and achieves joint optimization through an iterative solver, ultimately outputting a structurally complete 3D biological image. This method is applicable to biological sample scenarios where the image is acquired in blocks due to the limitations of the field of view in microscopic imaging, does not depend on a specific image modality, and can handle non-rigid deformation and noise interference.

[0033] For ease of understanding, the technical solution of the present invention will be described in detail with reference to the accompanying drawings. The drawings include: Figure 1 This is a flowchart of the overall solution. Figure 2 Example images for the training set; Figure 3 To optimize the model's principle block diagram; Figure 4 This is a schematic diagram of the constraint verification mechanism. The technical implementation of each part will be explained in detail below, with corresponding textual descriptions for each diagram.

[0034] The process of this invention is as follows: Figure 1 As shown. Figure 1 This is a flowchart, from top to bottom, including the following boxes: "Input multiple 3D biological image patches", "Construct stitching model", "Introduce global constraints", "Jointly optimize and solve pose", "Verify the validity of optimization results" (if the branch is "Yes", the stitching is completed; if "No", the constraints are adjusted and re-optimized), "Has the stitching goal been achieved?" (if the branch is "No", the model is updated and the loop continues; if "Yes", the loop terminates), and "Output the stitched 3D image". This flowchart uses a loop structure to represent the optimization process, ensuring that the algorithm iterates until the termination condition is met.

[0035] The detailed technical implementation is as follows: First, multiple three-dimensional biological image patches with initial spatial location information are acquired. These data are typically stored in TIFF or NIfTI format, with each patch being a local three-dimensional image patch with a resolution of 1 micrometer, derived from slice data imaging from an optical microscope. The initial pose is estimated using a preliminary registration algorithm (this invention is based on rigid registration maximizing mutual information): the transformation matrix is ​​calculated for adjacent patches. Initial translation and rotation parameters are obtained. A stitching model is constructed based on the adjacency relationships between image patches (determined through overlap detection, with a threshold overlap rate >10%): all image patches are abstracted as graph nodes, and the relationships between patches are edges, forming an undirected graph G=(V,E), where V is the patch pose variable (6 degrees of freedom: 3 translation + 3 rotation), and E is the relative constraint. Multiple global constraints are introduced (see the optimization model principle for details) to uniformly model the model, forming an energy function. .

[0036] By jointly optimizing the global constraint model, the optimal pose is solved using a least squares solver (Ceres Solver library): min arg E_total. The Levenberg-Marquardt algorithm is used iteratively with an adaptive step size (initial 1.0, decaying 0.8). Based on the optimization results, a validity verification is performed (see the verification mechanism section for details). If the verification passes, it checks whether the stitching target has been achieved (e.g., energy function change less than the threshold of 0.01 or the number of iterations exceeds 20). If not, the model is updated (constraint weights are adjusted), and a new model is built iteratively for further optimization. If the verification fails or the target is achieved, the algorithm terminates, and image patches are fused according to the optimal pose (using a weighted average to fuse overlapping areas). The weights are based on distance, and the output is a complete 3D biological image. The technical means of this process ensure global consistency of the stitching, reduce error accumulation through iterative optimization, and avoid drift problems in piece-by-piece stitching.

[0037] A key characteristic of 3D biological image patch stitching is the spatial continuity of biological tissues (such as nerve fibers or neuronal cell bodies) between adjacent patches. Therefore, the stitched result must maintain structural consistency with the original image patches; ensuring structural consistency is equivalent to guaranteeing the correctness of the stitched result. This invention utilizes this feature, employing an image patch generation algorithm based on simulated deformation to prepare a batch of training data (including 128 sets of patch data of mouse brain nerve fibers and 128 sets of patch data of mouse brain neuronal cell bodies) as the optimization constraint training set for the algorithm. Figure 2 As shown, the left image represents a nerve fiber, and the right image represents a neuron cell body. The blocks in the training set images were simulated by random cutting (each block is approximately 64x64x64 pixels in size), and the initial pose of each block was perturbed by noise. The quality of the labeled results (complete, unstitched images) of this training set was ensured after manual proofreading.

[0038] The detailed technical implementation is as follows: Training data preparation: First, complete 3D biological images of various complex scenes are collected. These data are from optical microscope imaging with a resolution of 1 micrometer. A random cutting algorithm is used to simulate block segmentation: a cutting plane is randomly selected to generate several image blocks, and initial pose noise (translation deviation ±5 pixels, rotation deviation ±5°) is applied. Each sample pair consists of (block image + initial pose, complete target image), totaling approximately 256 pairs. Data augmentation includes random deformation (thin plate spline transformation, 8*8*8 control points) and noise addition (Poisson noise, intensity 0.05) to generate diverse samples. Data preprocessing includes normalization: This ensures the input range is [0,1]. The data preparation techniques employed guarantee the model's robustness to deformation and noise, making it suitable for large-scale biological image scenarios.

[0039] This invention constructs a global optimization model to solve the image block pose. First, constraint signals are generated for the data in the training set. To describe the inter-block relationship, this invention performs preliminary feature matching on adjacent blocks, uses Scale-Invariant Feature Transform (SIFT) to extract key points, and calculates the relative pose, as shown in Equation (1): ,in , To match point pairs.

[0040] In optimization, global constraints are used to represent the overall state. Here, this invention uses energy terms to represent constraints. This invention constructs a graph optimization framework that takes an image patch and an initial pose as input to predict the optimal pose. Based on the constraints, this invention performs joint minimization of the pose, as shown in equation (2): ,in =0.5, =0.3.

[0041] like Figure 3 As shown, the algorithm principle block diagram includes the boxes for "input image patch and initial pose", "constructing graph model", "adding constraint terms", "solver optimization", and "outputting optimal pose". The diagram marks "global energy function" and "iterative solution". The detailed technical implementation is as follows: The optimization model used in this method is a graph optimization framework, which is divided into two main parts: graph construction and constraint addition. Graph construction uses the G2O (General Graph Optimization) library, with image patches as nodes, relative poses between patches as edges, and node variables as SE(3) transformations (special Euclidean group 3, representing pose). Constraint addition includes: vacancy suppression constraint. Minimize the gap volume by detecting pixel occupancy (threshold < 0.1 indicates a gap); overall smoothness constraint. Standardize the pose differences between adjacent blocks.

[0042] After the model is solved, the gradient is calculated using the Gauss-Newton method: Where J is the Jacobian matrix and r is the residual vector. Training uses simulated data to optimize the constraint weights: min L = MSE( , Adam optimizer, learning rate 1e-3, batch size 4, training for 500 epochs. This invention designs an adaptive weight strategy as shown in formula (3): * , where tau is the attenuation constant (10) to balance the constraints. After determining the parameters, the optimal pose can be solved, the image patch can be updated using the new pose, and the preceding process can be repeated to complete the global stitching. The technical means of this module make full use of multi-constraint cooperation to avoid local optima and is suitable for complex three-dimensional biological structures.

[0043] like Figure 4 As shown, in practical use, this invention allows the algorithm to optimize the stitching from the initial pose and records all constraint energy values. At each optimization step, it checks whether the similarity of the current pose is reasonable and uses reverse optimization for verification: applying the constraint model backwards from the current pose to determine whether the energy of the reverse optimization is reasonable compared to the error of the forward optimization, ensuring that the optimized pose always satisfies global consistency. Simultaneously, this invention also judges the relative relationship between the current stitching and neighboring blocks in real time, assessing stability based on factors such as whether the missing volume is minimized and whether the smoothness is consistent.

[0044] The complete images and segmented images in the training set of this invention have structural consistency. This invention utilizes this data to prepare the constraint information required by the algorithm for optimization training. This invention also introduces noise blocks that deviate from the pose into the dataset, and makes the noise blocks have large energy terms pointing to the original structure. With structurally consistent training data and appropriate noise data, the model learns a "global proximity" error correction strategy during training.

[0045] The detailed technical implementation is as follows: The verification mechanism includes a similarity check: calculating the NCC of the overlapping area after transformation; if NCC < 0.8, it is deemed unreasonable. Reverse optimization verification: starting from the current pose as a new starting point, the solver is iterated backward to generate the reverse pose, and the Frobenius norm error between the reverse and forward poses is calculated. If the error exceeds 0.05, the process is re-optimized. Stability is assessed using a gap volume threshold (<1% of total volume) and a smoothness index (pose difference variance <0.1). This mechanism employs multiple checks to ensure the biological validity of the stitching results and avoid artifacts or drift.

[0046] The key points of this invention are: 1. This invention models the problem of 3D biological image block stitching as a unified global constraint optimization problem, and solves it jointly by treating the spatial pose of each image block as an overall optimization variable, rather than using a block-by-block or sequential stitching method; 2. This invention introduces multiple constraints into the global stitching model to uniformly model the overlap relationship, gap situation and overall smoothness between image blocks, and achieve multi-constraint collaborative optimization; 3. This invention constrains the deformation of image blocks as a whole during the stitching process, effectively suppressing excessive local stretching or compression, and ensuring the authenticity and consistency of the spatial morphology of three-dimensional biological tissues; 4. This invention constructs a block stitching technology solution suitable for large-scale three-dimensional biological images, which can maintain the stability and global consistency of the stitching results when there are a large number of image blocks.

[0047] The points to be protected by this invention are: 1. The three-dimensional biological image block stitching method based on global constraint optimization proposed in this invention and its overall technical idea include a stitching process that uses the image block pose as a variable and performs joint optimization; 2. The global constraint model and its combination method established in this invention for image block overlap, gaps and overall smoothness are not limited to specific constraint forms or mathematical expressions; 3. The present invention provides a technical solution for controlling and constraining image block deformation during the process of three-dimensional biological image segmentation and stitching; 4. The system implementation scheme corresponding to the present invention includes the functional module division and its combination structure for implementing the three-dimensional biological image block stitching method.

[0048] Compared with existing methods for stitching together 3D biological images, which mainly employ local registration, sequential stitching, or block-by-block adjustment, this invention has the following advantages: 1. This invention models the block stitching of three-dimensional biological images as a global constraint optimization problem, and jointly solves the spatial pose of all image blocks, avoiding the problem of error accumulation block by block in the prior art, and significantly improving the global consistency and stability of the overall stitching result; 2. This invention introduces global constraints on image block overlap, gaps and overall smoothness during the stitching process, so that the stitching optimization target can simultaneously take into account local matching accuracy and overall structural continuity. Compared with existing methods that rely only on local features or adjacent relationships, it is more conducive to maintaining the true spatial morphology of biological tissues. 3. By uniformly controlling the deformation of image blocks through global optimization, this invention effectively suppresses the local stretching, compression or misalignment phenomena commonly found in existing technologies, and improves the reliability of the stitched 3D image in terms of structural scale and spatial proportion. 4. This invention can adapt to the block stitching requirements of large-scale three-dimensional biological images. Even when there are many image blocks and the structure is complex, it can still obtain stable stitching results. Compared with existing methods that rely on manual adjustment or multiple iterations, it reduces the cost of manual intervention and improves the overall processing efficiency. 5. Because the stitching results have higher consistency and continuity at the global level, this invention is beneficial for subsequent applications such as three-dimensional quantitative analysis, structural tracking and biological modeling, thereby enhancing the practical value of three-dimensional biological image data.

[0049] This invention has been experimentally verified and simulated on multiple sets of 3D biological image data. By applying the proposed global constraint optimization-based stitching method to actual acquired 3D biological image block data, the spatial consistency, structural continuity, and overall stability of the stitching results were verified. Experimental results show that this invention can effectively complete the overall stitching of 3D image blocks through global constraint optimization even with initial pose errors, reducing misalignment, voids, and local deformation problems during the stitching process, and obtaining 3D biological image results with structural continuity and spatial consistency.

[0050] Meanwhile, the technical solution of this invention has been applied and verified in actual three-dimensional biological image processing workflow. The results show that the method has good stability and repeatability, and can adapt to three-dimensional biological image block stitching tasks of different scales and complexities, proving that this invention has good feasibility and practical value in engineering applications.

[0051] Example 2 The system corresponding to this invention includes: a data input module for acquiring multiple 3D biological image patches; a model building module for constructing a stitching model based on adjacency relationships and introducing global constraints; an optimization solution module for jointly optimizing and solving the poses of the image patches; a verification module for checking the legality and stability of the optimization results; and an output module for fusing and outputting a complete 3D image based on the optimal pose. This system can be deployed on computing devices (such as servers equipped with graphics processors) and its functions are invoked through a software interface, ensuring that those skilled in the art can use it directly.

[0052] This invention provides a three-dimensional biological image block stitching system based on global constraint optimization. By modeling the three-dimensional biological image block stitching problem into a unified global optimization framework, the spatial pose of image blocks is jointly solved using multiple constraints, thereby achieving high-precision reconstruction of large-scale three-dimensional biological images and ensuring the continuity and consistency of the stitching results in spatial structure.

[0053] Specifically, this invention uses multiple local 3D biological image patches as input data, constructs a global stitching model using a graph optimization framework, treats the image patch pose as an optimization variable, introduces missing image suppression and smoothness constraints, and achieves joint optimization through an iterative solver, ultimately outputting a structurally complete 3D biological image. This method is applicable to biological sample scenarios where the image is acquired in blocks due to the limitations of the field of view in microscopic imaging, does not depend on a specific image modality, and can handle non-rigid deformation and noise interference.

[0054] For ease of understanding, the technical solution of the present invention will be described in detail with reference to the accompanying drawings. The drawings include: Figure 1 This is a flowchart of the overall solution. Figure 2 Example images for the training set; Figure 3 To optimize the model's principle block diagram; Figure 4 This is a schematic diagram of the constraint verification mechanism. The technical implementation of each part will be explained in detail below, with corresponding textual descriptions for each diagram.

[0055] The process of this invention is as follows: Figure 1 As shown. Figure 1 This is a flowchart, from top to bottom, including the following boxes: "Input multiple 3D biological image patches", "Construct stitching model", "Introduce global constraints", "Jointly optimize and solve pose", "Verify the validity of optimization results" (if the branch is "Yes", the stitching is completed; if "No", the constraints are adjusted and re-optimized), "Has the stitching goal been achieved?" (if the branch is "No", the model is updated and the loop continues; if "Yes", the loop terminates), and "Output the stitched 3D image". This flowchart uses a loop structure to represent the optimization process, ensuring that the algorithm iterates until the termination condition is met.

[0056] The detailed technical implementation is as follows: First, multiple three-dimensional biological image patches with initial spatial location information are acquired. These data are typically stored in TIFF or NIfTI format, with each patch being a local three-dimensional image patch with a resolution of 1 micrometer, derived from slice data imaging from an optical microscope. The initial pose is estimated using a preliminary registration algorithm (this invention is based on rigid registration maximizing mutual information): the transformation matrix is ​​calculated for adjacent patches. Initial translation and rotation parameters are obtained. A stitching model is constructed based on the adjacency relationships between image patches (determined through overlap detection, with a threshold overlap rate >10%): all image patches are abstracted as graph nodes, and the relationships between patches are edges, forming an undirected graph G=(V,E), where V is the patch pose variable (6 degrees of freedom: 3 translation + 3 rotation), and E is the relative constraint. Multiple global constraints are introduced (see the optimization model principle for details) to uniformly model the model, forming an energy function. .

[0057] By jointly optimizing the global constraint model, the optimal pose is solved using a least squares solver (Ceres Solver library): min arg E_total. The Levenberg-Marquardt algorithm is used iteratively with an adaptive step size (initial 1.0, decaying 0.8). Based on the optimization results, a validity verification is performed (see the verification mechanism section for details). If the verification passes, it checks whether the stitching target has been achieved (e.g., energy function change less than the threshold of 0.01 or the number of iterations exceeds 20). If not, the model is updated (constraint weights are adjusted), and a new model is built iteratively for further optimization. If the verification fails or the target is achieved, the algorithm terminates, and image patches are fused according to the optimal pose (using a weighted average to fuse overlapping areas). The weights are based on distance, and the output is a complete 3D biological image. The technical means of this process ensure global consistency of the stitching, reduce error accumulation through iterative optimization, and avoid drift problems in piece-by-piece stitching.

[0058] A key characteristic of 3D biological image patch stitching is the spatial continuity of biological tissues (such as nerve fibers or neuronal cell bodies) between adjacent patches. Therefore, the stitched result must maintain structural consistency with the original image patches; ensuring structural consistency is equivalent to guaranteeing the correctness of the stitched result. This invention utilizes this feature, employing an image patch generation algorithm based on simulated deformation to prepare a batch of training data (including 128 sets of patch data of mouse brain nerve fibers and 128 sets of patch data of mouse brain neuronal cell bodies) as the optimization constraint training set for the algorithm. Figure 2 As shown, the left image represents a nerve fiber, and the right image represents a neuron cell body. The blocks in the training set images were simulated by random cutting (each block is approximately 64x64x64 pixels in size), and the initial pose of each block was perturbed by noise. The quality of the labeled results (complete, unstitched images) of this training set was ensured after manual proofreading.

[0059] The detailed technical implementation is as follows: Training data preparation: First, complete 3D biological images of various complex scenes are collected. These data are from optical microscope imaging with a resolution of 1 micrometer. A random cutting algorithm is used to simulate block segmentation: a cutting plane is randomly selected to generate several image blocks, and initial pose noise (translation deviation ±5 pixels, rotation deviation ±5°) is applied. Each sample pair consists of (block image + initial pose, complete target image), totaling approximately 256 pairs. Data augmentation includes random deformation (thin plate spline transformation, 8*8*8 control points) and noise addition (Poisson noise, intensity 0.05) to generate diverse samples. Data preprocessing includes normalization: This ensures the input range is [0,1]. The data preparation techniques employed guarantee the model's robustness to deformation and noise, making it suitable for large-scale biological image scenarios.

[0060] This invention constructs a global optimization model to solve the image block pose. First, constraint signals are generated for the data in the training set. To describe the inter-block relationship, this invention performs preliminary feature matching on adjacent blocks, uses Scale-Invariant Feature Transform (SIFT) to extract key points, and calculates the relative pose, as shown in Equation (1): ,in , To match point pairs.

[0061] In optimization, global constraints are used to represent the overall state. Here, this invention uses energy terms to represent constraints. This invention constructs a graph optimization framework that takes an image patch and an initial pose as input to predict the optimal pose. Based on the constraints, this invention performs joint minimization of the pose, as shown in equation (2): ,in =0.5, =0.3.

[0062] like Figure 3 As shown, the algorithm principle block diagram includes the boxes for "input image patch and initial pose", "constructing graph model", "adding constraint terms", "solver optimization", and "outputting optimal pose". The diagram marks "global energy function" and "iterative solution". The detailed technical implementation is as follows: The optimization model used in this method is a graph optimization framework, which is divided into two main parts: graph construction and constraint addition. Graph construction uses the G2O (General Graph Optimization) library, with image patches as nodes, relative poses between patches as edges, and node variables as SE(3) transformations (special Euclidean group 3, representing pose). Constraint addition includes: vacancy suppression constraint. Minimize the gap volume by detecting pixel occupancy (threshold < 0.1 indicates a gap); overall smoothness constraint. Standardize the pose differences between adjacent blocks.

[0063] After the model is solved, the gradient is calculated using the Gauss-Newton method: Where J is the Jacobian matrix and r is the residual vector. Training uses simulated data to optimize the constraint weights: min L = MSE( , Adam optimizer, learning rate 1e-3, batch size 4, training for 500 epochs. This invention designs an adaptive weight strategy as shown in formula (3): * , where tau is the attenuation constant (10) to balance the constraints. After determining the parameters, the optimal pose can be solved, the image patch can be updated using the new pose, and the preceding process can be repeated to complete the global stitching. The technical means of this module make full use of multi-constraint cooperation to avoid local optima and is suitable for complex three-dimensional biological structures.

[0064] like Figure 4 As shown, in practical use, this invention allows the algorithm to optimize the stitching from the initial pose and records all constraint energy values. At each optimization step, it checks whether the similarity of the current pose is reasonable and uses reverse optimization for verification: applying the constraint model backwards from the current pose to determine whether the energy of the reverse optimization is reasonable compared to the error of the forward optimization, ensuring that the optimized pose always satisfies global consistency. Simultaneously, this invention also judges the relative relationship between the current stitching and neighboring blocks in real time, assessing stability based on factors such as whether the missing volume is minimized and whether the smoothness is consistent.

[0065] The complete images and segmented images in the training set of this invention have structural consistency. This invention utilizes this data to prepare the constraint information required by the algorithm for optimization training. This invention also introduces noise blocks that deviate from the pose into the dataset, and makes the noise blocks have large energy terms pointing to the original structure. With structurally consistent training data and appropriate noise data, the model learns a "global proximity" error correction strategy during training.

[0066] The detailed technical implementation is as follows: The verification mechanism includes a similarity check: calculating the NCC of the overlapping area after transformation; if NCC < 0.8, it is deemed unreasonable. Reverse optimization verification: starting from the current pose as a new starting point, the solver is iterated backward to generate the reverse pose, and the Frobenius norm error between the reverse and forward poses is calculated. If the error exceeds 0.05, the process is re-optimized. Stability is assessed using a gap volume threshold (<1% of total volume) and a smoothness index (pose difference variance <0.1). This mechanism employs multiple checks to ensure the biological validity of the stitching results and avoid artifacts or drift.

[0067] The key points of this invention are: 1. This invention models the problem of 3D biological image block stitching as a unified global constraint optimization problem, and solves it jointly by treating the spatial pose of each image block as an overall optimization variable, rather than using a block-by-block or sequential stitching method; 2. This invention introduces multiple constraints into the global stitching model to uniformly model the overlap relationship, gap situation and overall smoothness between image blocks, and achieve multi-constraint collaborative optimization; 3. This invention constrains the deformation of image blocks as a whole during the stitching process, effectively suppressing excessive local stretching or compression, and ensuring the authenticity and consistency of the spatial morphology of three-dimensional biological tissues; 4. This invention constructs a block stitching technology solution suitable for large-scale three-dimensional biological images, which can maintain the stability and global consistency of the stitching results when there are a large number of image blocks.

[0068] The points to be protected by this invention are: 1. The three-dimensional biological image block stitching method based on global constraint optimization proposed in this invention and its overall technical idea include a stitching process that uses the image block pose as a variable and performs joint optimization; 2. The global constraint model and its combination method established in this invention for image block overlap, gaps and overall smoothness are not limited to specific constraint forms or mathematical expressions; 3. The present invention provides a technical solution for controlling and constraining image block deformation during the process of three-dimensional biological image segmentation and stitching; 4. The system implementation scheme corresponding to the present invention includes the functional module division and its combination structure for implementing the three-dimensional biological image block stitching method.

[0069] Compared with existing methods for stitching together 3D biological images, which mainly employ local registration, sequential stitching, or block-by-block adjustment, this invention has the following advantages: 1. This invention models the block stitching of three-dimensional biological images as a global constraint optimization problem, and jointly solves the spatial pose of all image blocks, avoiding the problem of error accumulation block by block in the prior art, and significantly improving the global consistency and stability of the overall stitching result; 2. This invention introduces global constraints on image block overlap, gaps and overall smoothness during the stitching process, so that the stitching optimization target can simultaneously take into account local matching accuracy and overall structural continuity. Compared with existing methods that rely only on local features or adjacent relationships, it is more conducive to maintaining the true spatial morphology of biological tissues. 3. By uniformly controlling the deformation of image blocks through global optimization, this invention effectively suppresses the local stretching, compression or misalignment phenomena commonly found in existing technologies, and improves the reliability of the stitched 3D image in terms of structural scale and spatial proportion. 4. This invention can adapt to the block stitching requirements of large-scale three-dimensional biological images. Even when there are many image blocks and the structure is complex, it can still obtain stable stitching results. Compared with existing methods that rely on manual adjustment or multiple iterations, it reduces the cost of manual intervention and improves the overall processing efficiency. 5. Because the stitching results have higher consistency and continuity at the global level, this invention is beneficial for subsequent applications such as three-dimensional quantitative analysis, structural tracking and biological modeling, thereby enhancing the practical value of three-dimensional biological image data.

[0070] This invention has been experimentally verified and simulated on multiple sets of 3D biological image data. By applying the proposed global constraint optimization-based stitching method to actual acquired 3D biological image block data, the spatial consistency, structural continuity, and overall stability of the stitching results were verified. Experimental results show that this invention can effectively complete the overall stitching of 3D image blocks through global constraint optimization even with initial pose errors, reducing misalignment, voids, and local deformation problems during the stitching process, and obtaining 3D biological image results with structural continuity and spatial consistency.

[0071] Meanwhile, the technical solution of this invention has been applied and verified in actual three-dimensional biological image processing workflow. The results show that the method has good stability and repeatability, and can adapt to three-dimensional biological image block stitching tasks of different scales and complexities, proving that this invention has good feasibility and practical value in engineering applications.

[0072] Example 3 A storage medium storing a program file capable of implementing any of the above-mentioned three-dimensional biological image block stitching methods based on global constraint optimization.

[0073] Example 4 A processor for running a program, wherein the program executes any of the above-mentioned three-dimensional biological image block stitching methods based on global constraint optimization.

[0074] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0075] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0076] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The system embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0077] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0078] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0079] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0080] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for segmenting and stitching three-dimensional biological images based on global constraint optimization, characterized in that, include: Acquire multiple 3D biological image patches with initial spatial location information; A stitching model is constructed based on the adjacency relationship between 3D biological image blocks, and the spatial pose of each 3D biological image block is used as an optimization variable. Multiple global constraints are introduced into the stitching model to uniformly model the overlap relationship, gap situation and overall smoothness between 3D biological image blocks; By jointly optimizing the global constraints, the optimal spatial pose parameters of each 3D biological image block are solved, and the overall stitching and reconstruction of the 3D biological image is completed accordingly, resulting in a 3D biological image with consistent spatial structure and continuous morphology.

2. The three-dimensional biological image block stitching method based on global constraint optimization according to claim 1, characterized in that, A stitching model is constructed based on the adjacency relationships between 3D biological image patches, and the spatial pose of each 3D biological image patch is used as an optimization variable, including: A stitching model is constructed by obtaining the adjacency relationship between image patches through overlap detection. Rigid registration based on maximizing mutual information and initial pose estimation using a preliminary registration algorithm.

3. The three-dimensional biological image block stitching method based on global constraint optimization according to claim 2, characterized in that, Building a stitching model by obtaining the adjacency relationship between image patches through overlap detection includes: All image blocks are abstracted as graph nodes, and the relationships between blocks are edges, forming an undirected graph.

4. The three-dimensional biological image block stitching method based on global constraint optimization according to claim 2, characterized in that, Rigid registration based on maximizing mutual information and estimation of the initial pose using a preliminary registration algorithm includes: Calculate the transformation matrix for adjacent blocks to obtain the initial translation and rotation parameters.

5. The three-dimensional biological image block stitching method based on global constraint optimization according to claim 1, characterized in that, Multiple global constraints are introduced into the stitching model to uniformly model the overlap relationship, gaps, and overall smoothness between 3D biological image patches, including: Multiple global constraints are introduced to perform unified modeling of the splicing model, forming an energy function; The optimal pose is solved by jointly optimizing the global constraint model and using a least squares solver.

6. The three-dimensional biological image block stitching method based on global constraint optimization according to claim 1, characterized in that, Also includes: Based on the optimization results, a legality verification is performed. If the verification passes, the stitching target is checked. If the target is not achieved, the model is updated, the constraint weights are adjusted, and a new model is built in a loop to continue optimization. If the verification fails or the target is achieved, the algorithm is terminated, and the image patch is fused according to the optimal pose to output a complete 3D biological image.

7. The three-dimensional biological image block stitching method based on global constraint optimization according to claim 1, characterized in that, Also includes: A batch of training data is prepared using an image patch generation algorithm based on simulated deformation as the training set for the algorithm's optimization constraints.

8. The three-dimensional biological image block stitching method based on global constraint optimization according to claim 7, characterized in that, Introduce noise blocks that deviate from the pose into the dataset, and make the noise blocks have energy terms pointing to the original structure. With training data that is structurally consistent and appropriate noise data, the model training can achieve a global error correction function.

9. The three-dimensional biological image block stitching method based on global constraint optimization according to claim 1, characterized in that, Also includes: The stitching is optimized starting from the initial pose, and all constraint energy values ​​are recorded. At each optimization step, the similarity of the current pose is checked for reasonableness, and back-optimization is used for verification. By applying the constraint model backward from the current pose, we can determine whether the energy of the backward optimization is reasonable compared with the error of the forward optimization, and ensure that the optimized pose always satisfies global consistency. The system determines the relative relationship between the current patch and its neighboring blocks in real time, and judges whether it is stable by whether the missing volume is minimized and whether the smoothness is consistent.

10. A three-dimensional biological image block stitching system based on global constraint optimization, characterized in that, include: The data input module is used to acquire multiple 3D biological image blocks with initial spatial location information; The model building module is used to construct a stitching model based on the adjacency relationship between 3D biological image patches, and the spatial pose of each 3D biological image patch is used as an optimization variable. The optimization solution module is used to introduce various global constraints into the stitching model and to uniformly model the overlap relationship, gap situation and overall smoothness between 3D biological image blocks; The output module is used to jointly optimize global constraints, solve for the optimal spatial pose parameters of each 3D biological image block, and complete the overall stitching and reconstruction of the 3D biological image to obtain a 3D biological image with consistent spatial structure and continuous morphology.

Images