Three-dimensional biological image cross-modality registration method based on domain generalization representation learning

Abstract

The application relates to a three-dimensional biological image cross-modal registration method based on domain generalization representation learning, which comprises the following steps: obtaining a target image feature point set to obtain an initial point set of a to-be-registered image; obtaining a first feature block and a second feature block; extracting domain generalization features and calculating the similarity score between the first feature block and the second feature block; calculating the scores of all search points within a search distance, and then calculating the displacement of the initial point set of the to-be-registered image from the scores of all the search points; obtaining a new feature point set on the to-be-registered image, and returning to step (2) for optimization until the initial point set of the to-be-registered image finds the best matching position with the target image feature point set to obtain the optimal matching point set of the to-be-registered image; and deforming the to-be-registered image to obtain a registered three-dimensional biological image. The application can realize seamless switching between different modal images, can register higher-resolution microscopic images, and thus improves the registration accuracy and availability.

Description

Technical Field

[0001] This invention relates to the field of 3D biomedical image registration technology, and in particular to a cross-modal registration method for three-dimensional biological images based on domain generalization representation learning. Background Technology

[0002] In the field of medical imaging today, cross-modal registration of three-dimensional biological images is a challenging task. Different imaging systems provide images of the mouse brain in various modalities, encompassing macroscopic magnetic resonance imaging (MRI), microscopic electron microscopy (EM), and mesoscopic methods such as STPT, fMOST, LSFM, and VISoR. These different imaging modalities typically require various sample preparation methods (such as solidification, embedding, and clearing) and imaging through laser scanning, mechanical sectioning, and optical sectioning. Different staining and labeling techniques (such as Nissl, DAPI, transgenic, and viral) are also applied during image acquisition. These different sample preparation, imaging methods, and staining techniques result in significant differences in scale, brightness, texture, and anatomical structure among the brain imaging data from different modalities, making cross-modal brain registration a highly challenging task.

[0003] Currently, research on cross-modal registration of 3D biological images mainly covers two main approaches:

[0004] Firstly, traditional image registration methods are employed, utilizing classic computer vision techniques such as feature matching and optimization. These methods typically rely on the geometric and grayscale information of the image, such as feature point extraction and matching, and least squares methods, to achieve image alignment through optimization of the registration model. However, these methods are limited in their performance when dealing with challenges such as different modalities, large scale variations, and complex structures.

[0005] Secondly, there are deep learning-based registration algorithms. In recent years, deep learning has made significant progress in the field of image registration. Deep learning methods, by constructing end-to-end neural networks, can learn feature representations from large-scale data, achieving higher-precision registration results. Deep learning architectures such as Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) are widely used in image registration tasks, providing effective registration solutions for cross-modal, large-scale, and complex scenes. However, due to the limitations of computer processing power and memory, deep networks cannot support large-size image inputs. Although image size can be reduced by downsampling, excessive downsampling will result in the loss of a large number of image features, making it impossible to obtain high-precision registration results. Summary of the Invention

[0006] To address the issue of low accuracy in cross-modal registration of 3D biological images using existing technologies, this invention aims to provide a cross-modal registration method for 3D biological images based on domain generalization representation learning that achieves more accurate registration results and exhibits higher reliability and robustness in practical applications.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a cross-modal registration method for three-dimensional biological images based on domain generalization representation learning, the method comprising the following sequential steps:

[0008] (1) Obtain the feature point set of the target image, apply prior constraints to the feature point set of the target image, copy the feature point set of the target image and map it onto the image to be registered to obtain the initial point set of the image to be registered.

[0009] (2) Extract feature blocks on the target image based on the feature point set of the target image to obtain the first feature block. With the initial point set of the image to be registered as the center and the variable r as the search distance (r is a positive integer), extract feature blocks on the image to be registered based on the search points within the search distance to obtain the second feature block.

[0010] (3) Input the first feature block and the second feature block into the trained deep neural network in pairs, extract the domain generalization features, and calculate the similarity score between the first feature block and the second feature block;

[0011] (4) Calculate the score of all search points within the search distance based on the similarity score between the first feature block and the second feature block, and then calculate the displacement of the initial point set of the image to be registered based on the scores of all search points;

[0012] (5) Obtain a new set of feature points on the image to be registered by the displacement of the initial point set of the image to be registered, and then return to step (2) for optimization until the initial point set of the image to be registered finds the best matching position with the feature point set of the target image, and obtain the optimal matching point set of the image to be registered.

[0013] (6) Based on the optimal matching point set between the feature point set of the target image and the image to be registered, the deformation field is calculated by the STPS algorithm, and the image to be registered is deformed to obtain the registered three-dimensional biological image.

[0014] Step (1) specifically includes the following steps:

[0015] (1a) Read the target image and obtain the corresponding contour map and label map;

[0016] (1b) Read the feature point set of the target image, add corresponding prior constraint information according to whether the feature point is on the contour, and add corresponding prior constraint information according to the label pixel value of the local area corresponding to the label map where the feature point is located.

[0017] (1c) Copy the feature point set of the target image and map it directly onto the image to be registered to obtain the initial point set of the image to be registered.

[0018] Step (2) specifically includes the following steps:

[0019] (2a) Centered on the feature point set of the target image, extract the first feature block with a size of 21×21×21 on the target image, and extract a total of T features on the target image. ind The first feature block, T ind It also refers to the number of feature points in the target image;

[0020] (2b) On the image to be registered, with the initial point set of the image to be registered as the center and the variable r as the search distance, then each feature point has P = (2*r+1) within the search distance. 3 Using 1 search point, a second feature block of size 21×21×21 is extracted from the image to be registered. A total of S features are extracted from the image to be registered. ind ×S s There are two second feature blocks, where S ind S represents the number of initial point sets in the image to be registered, and S ind =T ind S s The number of search points included for each feature point.

[0021] In step (3), the trained deep neural network is obtained through the following steps:

[0022] (3a1) Obtain whole-brain images of 30 mice with fMOST modality at 25µm resolution, 30 mice with VISoR modality at 25µm resolution, and 30 mice with STPT modality at 25µm resolution. Use the mBrainAligner algorithm to register the mouse brains of the fMOST modality to the fMOST mouse brain mean template, the mouse brains of the VISoR modality to the VISoR mouse brain mean template, and the mouse brains of the STPT modality to the STPT mouse brain mean template. Then, perform semi-automatic optimization on all mouse brains. Register the mouse brain mean templates of fMOST and VISoR to the STPT mouse brain mean template.

[0023] (3a2) Take 3700 points in the main texture and contour of the STPT mouse brain mean template, and then use the STPS algorithm to inversely transform these points to the mouse brain of each modality, so that the mouse brain structural information of the same numbered points on each mouse brain of each modality corresponds.

[0024] (3a3) Using the points on each mouse brain as the center, extract image patches of size 21×21×21 on the mouse brain as the dataset. Randomly select 80% of all data in the dataset as the training set and 20% as the validation set.

[0025] (3a4) Construct a deep neural network, the deep neural network comprising:

[0026] The feature extraction module is used to extract features from the input first feature block and second feature block to obtain the first target feature map and the first feature map to be registered.

[0027] The domain generalization module is used to extract features from the input first target feature map and first registration feature map to obtain the second target feature map and the second registration feature map.

[0028] The metric module is used to measure and calculate the L1 distance between the input second target feature map and the second feature map to be registered, and to obtain the similarity score between the first feature block and the second feature block.

[0029] (3a5) During training, two image blocks are randomly selected from the training set as the first feature block and the second feature block. Then, the first feature block and the second feature block are augmented with data and fed into the deep neural network for similarity calculation.

[0030] In step (3), calculating the similarity score between the first feature block and the second feature block specifically refers to: extracting the first feature block T from the target image ind [i] and the second feature block S extracted from the image to be registered ind [i]×S s [j] are sequentially input into the trained deep neural network to predict similarity, resulting in a similarity score Similarity[i][j]; where T ind [i] represents the first feature block of size 21×21×21 extracted from the i-th point of the target image, S ind [i]×S s [j] represents the second feature block of size 21×21×21 extracted from the j-th search point within the search range of the i-th feature point in the image to be registered, and Similarity[i][j] represents the similarity score between the i-th point in the target image and the j-th search point within the search range of the i-th point in the image to be registered.

[0031] Step (4) specifically includes the following steps:

[0032] (4a) Calculate the Euclidean distance between the initial point set and the search point of the image to be registered, and store it in the matrix dis_map. ind In the middle, dis_map ind[i][j] represents the Euclidean distance between the j-th search point within the search range of the i-th feature point in the image to be registered and the feature point itself, dis_map ind If the size is P, then the formula for calculating the search point score is as follows:

[0033]

[0034] Where Score[i][j] represents the score of the j-th search point within the search range of the i-th feature point in the image to be registered;

[0035] (4b) Calculate the displacement of the initial point set of the image to be registered. The calculation formula is as follows:

[0036]

[0037]

[0038]

[0039]

[0040] dis_x[i]=d_sum_x[i] / d_sum[i]

[0041] dis_y[i]=d_sum_y[i] / d_sum[i]

[0042] dis_z[i]=d_sum_z[i] / d_sum[i]

[0043] Among them, dis_map ind [i][j]_x represents the x-direction displacement between the j-th search point and the i-th feature point within the search range of the i-th feature point in the image to be registered; dis_map ind [i][j]_y represents the displacement in the y-direction between the j-th search point within the search range of the i-th feature point in the image to be registered and that feature point; dis_map ind [i][j]_z represents the displacement in the z-direction between the j-th search point within the search range of the i-th feature point in the image to be registered and that feature point; dis_x[i] is the calculated displacement in the x-direction of the initial point set of the image to be registered; dis_y[i] is the calculated displacement in the y-direction of the initial point set of the image to be registered; and dis_z[i] is the calculated displacement in the z-direction of the initial point set of the image to be registered.

[0044] Step (5) specifically includes the following steps:

[0045] (5a) The new feature point set sub_new_markers on the image to be registered is obtained from the displacement of the initial point set of the image to be registered. The calculation formula is as follows:

[0046] sub_new_markers_x[i]=sub_markers_x[i]+dis_x[i]

[0047] sub_new_markers_y[i]=sub_markers_y[i]+dis_y[i]

[0048] sub_new_markers_z[i]=sub_markers_z[i]+dis_z[i]

[0049] Where, sub_new_markers_x[i] represents the x-coordinate of the new i-th feature point on the image to be registered, sub_new_markers_y[i] represents the y-coordinate of the new i-th feature point on the image to be registered, sub_new_markers_z[i] represents the z-coordinate of the new i-th feature point on the image to be registered; sub_markers_x[i] represents the x-coordinate of the i-th feature point on the image to be registered before displacement, sub_markers_y[i] represents the y-coordinate of the i-th feature point on the image to be registered before displacement, sub_markers_z[i] represents the z-coordinate of the i-th feature point on the image to be registered before displacement; dis_x[i] is the calculated displacement of the initial point set of the image to be registered in the x-direction; dis_y[i] is the calculated displacement of the initial point set of the image to be registered in the y-direction; dis_z[i] is the calculated displacement of the initial point set of the image to be registered in the z-direction;

[0050] (5b) Draw a new search range from the new feature point set on the image to be registered, and re-infer the similarity between the new feature blocks. That is, return to step (2) iteration, and replace the initial point set of the image to be registered with the new feature point set on the image to be registered to obtain the second feature block. Until the displacements dis_x[i], dis_y[i], and dis_z[i] of the feature points on the image to be registered in step (5) are all less than 0.1 for 10 consecutive times, the iteration ends and enters step (6).

[0051] As can be seen from the above technical solution, the beneficial effects of the present invention are as follows: First, the present invention performs better in cross-modal registration of three-dimensional biological images, and can more accurately capture the complex relationships between images of different modalities, achieving more accurate registration results. In the fields of multimodal medical image registration, the present invention has higher reliability and robustness in practical applications. Second, the present invention can be used for both intramodal and cross-modal registration, and can achieve seamless switching between images of different modalities, providing a unified and efficient solution for a wide range of medical image registration tasks. Third, compared with other depth-based registration algorithms, the present invention can register higher resolution microscopic images, thereby improving the accuracy and usability of registration. Attached Figure Description

[0052] Figure 1 This is a flowchart of the method of the present invention;

[0053] Figure 2 This is a structural diagram of the deep neural network in this invention. Detailed Implementation

[0054] like Figure 1 As shown, a cross-modal registration method for 3D biological images based on domain generalization representation learning includes the following sequential steps:

[0055] (1) Obtain the feature point set of the target image, apply prior constraints to the feature point set of the target image, copy the feature point set of the target image and map it onto the image to be registered to obtain the initial point set of the image to be registered.

[0056] (2) Extract feature blocks on the target image based on the feature point set of the target image to obtain the first feature block. With the initial point set of the image to be registered as the center and the variable r as the search distance (r is a positive integer), extract feature blocks on the image to be registered based on the search points within the search distance to obtain the second feature block.

[0057] (3) Input the first feature block and the second feature block into the trained deep neural network in pairs, extract the domain generalization features, and calculate the similarity score between the first feature block and the second feature block;

[0058] (4) Calculate the score of all search points within the search distance based on the similarity score between the first feature block and the second feature block, and then calculate the displacement of the initial point set of the image to be registered based on the scores of all search points;

[0059] (5) Obtain a new set of feature points on the image to be registered by the displacement of the initial point set of the image to be registered, and then return to step (2) for optimization until the initial point set of the image to be registered finds the best matching position with the feature point set of the target image, and obtain the optimal matching point set of the image to be registered.

[0060] (6) Based on the optimal matching point set between the feature point set of the target image and the image to be registered, the deformation field is calculated by the STPS algorithm, and the image to be registered is deformed to obtain the registered three-dimensional biological image.

[0061] Step (1) specifically includes the following steps:

[0062] (1a) Read the target image and obtain the corresponding contour map and label map;

[0063] (1b) Read the feature point set of the target image, add corresponding prior constraint information according to whether the feature point is on the contour, and add corresponding prior constraint information according to the label pixel value of the local area corresponding to the label map where the feature point is located.

[0064] (1c) Copy the feature point set of the target image and map it directly onto the image to be registered to obtain the initial point set of the image to be registered.

[0065] Step (2) specifically includes the following steps:

[0066] (2a) Centered on the feature point set of the target image, extract the first feature block with a size of 21×21×21 on the target image, and extract a total of T features on the target image. ind The first feature block, T ind It also refers to the number of feature points in the target image;

[0067] (2b) On the image to be registered, with the initial point set of the image to be registered as the center and the variable r as the search distance, then each feature point has P = (2*r+1) within the search distance. 3 Using 1 search point, a second feature block of size 21×21×21 is extracted from the image to be registered. A total of S features are extracted from the image to be registered. ind ×S s There are two second feature blocks, where S ind S represents the number of initial point sets in the image to be registered, and S ind =T ind S s The number of search points included for each feature point.

[0068] In step (3), the trained deep neural network is obtained through the following steps:

[0069] (3a1) Obtain whole-brain data of 30 mice with 25µm resolution from the fMOST modality, 30 mice with 25µm resolution from the VISoR modality, and 30 mice with 25µm resolution from the STPT modality. Use the mBrainAligner algorithm to register the mouse brains of the fMOST modality to the fMOST mouse brain mean template, the mouse brains of the VISoR modality to the VISoR mouse brain mean template, and the mouse brains of the STPT modality to the STPT mouse brain mean template. Then, perform semi-automatic optimization on all mouse brains. Register the mouse brain mean templates of the fMOST and VISoR mouse brains to the STPT mouse brain mean template to break down the barriers between these modal data and enable them to be comprehensively analyzed in a unified common space to obtain a cross-modal dataset.

[0070] (3a2) Take 3700 points in the main texture and contour of the STPT mouse brain mean template, and then use the STPS algorithm to inversely transform these points to the mouse brain of each modality, so that the mouse brain structural information of the same numbered points on each mouse brain of each modality corresponds.

[0071] (3a3) Using the points on each mouse brain as the center, extract image patches of size 21×21×21 on the mouse brain as the dataset. Randomly select 80% of all data in the dataset as the training set and 20% as the validation set.

[0072] (3a4) Construct a deep neural network, such as Figure 2 As shown, the deep neural network includes:

[0073] The feature extraction module is used to extract features from the input first feature block and second feature block to obtain the first target feature map and the first feature map to be registered.

[0074] The domain generalization module is used to extract features from the input first target feature map and first registration feature map to obtain the second target feature map and the second registration feature map.

[0075] The metric module is used to measure and calculate the L1 distance between the input second target feature map and the second feature map to be registered, and to obtain the similarity score between the first feature block and the second feature block.

[0076] (3a5) During training, two image blocks are randomly selected from the training set as the first feature block and the second feature block. Then, the first feature block and the second feature block are augmented with data and fed into the deep neural network for similarity calculation.

[0077] In step (3), calculating the similarity score between the first feature block and the second feature block specifically refers to: extracting the first feature block T from the target image ind[i] and the second feature block S extracted from the image to be registered ind [i]×S s [j] are sequentially input into the trained deep neural network to predict similarity, resulting in a similarity score Similarity[i][j]; where T ind [i] represents the first feature block of size 21×21×21 extracted from the i-th point of the target image, S ind [i]×S s [j] represents the second feature block of size 21×21×21 extracted from the j-th search point within the search range of the i-th feature point in the image to be registered, and Similarity[i][j] represents the similarity score between the i-th point in the target image and the j-th search point within the search range of the i-th point in the image to be registered.

[0078] Step (4) specifically includes the following steps:

[0079] (4a) Calculate the Euclidean distance between the initial point set and the search point of the image to be registered, and store it in the matrix dis_map. ind In the middle, dis_map ind [i][j] represents the Euclidean distance between the j-th search point within the search range of the i-th feature point in the image to be registered and the feature point itself, dis_map ind If the size is P, then the formula for calculating the search point score is as follows:

[0080]

[0081] Where Score[i][j] represents the score of the j-th search point within the search range of the i-th feature point in the image to be registered;

[0082] (4b) Calculate the displacement of the initial point set of the image to be registered. The calculation formula is as follows:

[0083]

[0084]

[0085]

[0086]

[0087] dis_x[i]=d_sum_x[i] / d_sum[i]

[0088] dis_y[i]=d_sum_y[i] / d_sum[i]

[0089] dis_z[i]=d_sum_z[i] / d_sum[i]

[0090] Among them, dis_map ind [i][j]_x represents the x-direction displacement between the j-th search point and the i-th feature point within the search range of the i-th feature point in the image to be registered; dis_map ind [i][j]_y represents the displacement in the y-direction between the j-th search point within the search range of the i-th feature point in the image to be registered and that feature point; dis_map ind [i][j]_z represents the displacement in the z-direction between the j-th search point within the search range of the i-th feature point in the image to be registered and that feature point; dis_x[i] is the calculated displacement in the x-direction of the initial point set of the image to be registered; dis_y[i] is the calculated displacement in the y-direction of the initial point set of the image to be registered; and dis_z[i] is the calculated displacement in the z-direction of the initial point set of the image to be registered.

[0091] Step (5) specifically includes the following steps:

[0092] (5a) The new feature point set sub_new_markers on the image to be registered is obtained from the displacement of the initial point set of the image to be registered. The calculation formula is as follows:

[0093] sub_new_markers_x[i]=sub_markers_x[i]+dis_x[i]

[0094] sub_new_markers_y[i]=sub_markers_y[i]+dis_y[i]

[0095] sub_new_markers_z[i]=sub_markers_z[i]+dis_z[i]

[0096] Where, sub_new_markers_x[i] represents the x-coordinate of the new i-th feature point on the image to be registered, sub_new_markers_y[i] represents the y-coordinate of the new i-th feature point on the image to be registered, sub_new_markers_z[i] represents the z-coordinate of the new i-th feature point on the image to be registered; sub_markers_x[i] represents the x-coordinate of the i-th feature point on the image to be registered before displacement, sub_markers_y[i] represents the y-coordinate of the i-th feature point on the image to be registered before displacement, sub_markers_z[i] represents the z-coordinate of the i-th feature point on the image to be registered before displacement; dis_x[i] is the calculated displacement of the initial point set of the image to be registered in the x-direction; dis_y[i] is the calculated displacement of the initial point set of the image to be registered in the y-direction; dis_z[i] is the calculated displacement of the initial point set of the image to be registered in the z-direction;

[0097] (5b) Draw a new search range from the new feature point set on the image to be registered, and re-infer the similarity between the new feature blocks. That is, return to step (2) iteration, and replace the initial point set of the image to be registered with the new feature point set on the image to be registered to obtain the second feature block. Until the displacements dis_x[i], dis_y[i], and dis_z[i] of the feature points on the image to be registered in step (5) are all less than 0.1 for 10 consecutive times, the iteration ends and enters step (6).

[0098] like Figure 2 As shown, patch1 is the first feature block, and patch2 is the second feature block. The first and second feature blocks are input to the feature extraction module and the domain generalization module to extract the second target feature map and the second feature map to be registered. The L1 distance between the second target feature map and the second feature map to be registered is calculated and then sent to the metric module. The metric module uses a fully connected layer, which consists of a first fully connected layer, a first dropout layer, a second fully connected layer, a second dropout layer, a third fully connected layer, a third dropout layer, and a neuron. The value is converted into a probability value by the sigmoid activation function, representing the similarity score between the input first feature block and the second feature block.

[0099] Table 1 shows a comparison of the registration accuracy of this invention with that of the mBrainAligner algorithm. Compared to the mBrainAligner algorithm, this invention achieves higher accuracy in both mono-modal and cross-modal registration. Specifically, in cross-modal registration, this invention demonstrates higher accuracy in the HPF, CTX, CBX, and CP brain regions, while in mono-modal registration, it exhibits higher accuracy in the TH, HPF, MB, and CP brain regions.

[0100] Table 1

[0101]

[0102] In summary, this invention performs superiorly in cross-modal registration of 3D biological images, more accurately capturing the complex relationships between images of different modalities and achieving more precise registration results. In fields such as multimodal medical image registration, this invention demonstrates higher reliability and robustness in practical applications. This invention can be used simultaneously for both intramodal and cross-modal registration, enabling seamless switching between images of different modalities, providing a unified and efficient solution for a wide range of medical image registration tasks. Compared to other depth-based registration algorithms, this invention can register higher-resolution microscopic images, thereby improving the accuracy and usability of registration.

Claims

1. A cross-modal registration method for 3D biological images based on domain generalization representation learning, characterized in that: The method includes the following steps in sequence: (1) Obtain the feature point set of the target image, apply prior constraints to the feature point set of the target image, copy the feature point set of the target image and map it onto the image to be registered to obtain the initial point set of the image to be registered; (2) Extract feature blocks on the target image based on the feature point set of the target image to obtain the first feature block. With the initial point set of the image to be registered as the center and the variable r as the search distance (r is a positive integer), extract feature blocks on the image to be registered based on the search points within the search distance to obtain the second feature block. (3) Input the first feature block and the second feature block into the trained deep neural network in pairs, extract the domain generalization features, and calculate the similarity score between the first feature block and the second feature block; (4) Calculate the score of all search points within the search distance based on the similarity score between the first feature block and the second feature block, and then calculate the displacement of the initial point set of the image to be registered based on the scores of all search points; Step (4) specifically includes the following steps: (4a) Calculate the Euclidean distance between the initial point set of the image to be registered and the search point, and store it in a matrix. middle, This represents the Euclidean distance between the j-th search point within the search range of the i-th feature point in the image to be registered and that feature point. If the size is P, then the formula for calculating the search point score is as follows: ； in, This represents the score of the j-th search point within the search range of the i-th feature point in the image to be registered; Similarity score; (4b) Calculate the displacement of the initial point set of the image to be registered. The calculation formula is as follows: ； ； ； ； ； ； ； in, This represents the displacement in the x-direction between the j-th search point within the search range of the i-th feature point in the image to be registered and that feature point; This represents the displacement in the y-direction between the j-th search point within the search range of the i-th feature point in the image to be registered and that feature point; This represents the displacement in the z-direction between the j-th search point within the search range of the i-th feature point in the image to be registered and that feature point; The displacement of the initial point set of the image to be registered in the x-direction is calculated. The displacement of the initial point set of the image to be registered in the y direction is calculated. The displacement of the initial point set of the image to be registered in the z-direction is calculated. (5) Obtain a new set of feature points on the image to be registered by the displacement of the initial point set of the image to be registered, and then return to step (2) for optimization until the initial point set of the image to be registered finds the best matching position with the feature point set of the target image, and obtain the optimal matching point set of the image to be registered. (6) Based on the optimal matching point set between the feature point set of the target image and the image to be registered, the deformation field is calculated by the STPS algorithm, and the image to be registered is deformed to obtain the registered three-dimensional biological image.

2. The method for cross-modal registration of three-dimensional biological images based on domain generalization representation learning according to claim 1, characterized in that: Step (1) specifically includes the following steps: (1a) Read the target image and obtain the corresponding contour map and label map; (1b) Read the feature point set of the target image, add corresponding prior constraint information according to whether the feature point is on the contour, and add corresponding prior constraint information according to the label pixel value of the local area corresponding to the label map where the feature point is located; (1c) Copy the feature point set of the target image and map it directly onto the image to be registered to obtain the initial point set of the image to be registered.

3. The method for cross-modal registration of 3D biological images based on domain generalization representation learning according to claim 1, characterized in that: Step (2) specifically includes the following steps: (2a) Using the feature point set of the target image as the center, extract the feature points of size on the target image. The first feature block is extracted from the target image. The first feature block, It also refers to the number of feature points in the target image; (2b) On the image to be registered, with the initial point set of the image to be registered as the center and the variable r as the search distance, then each feature point has within the search distance... Search points are used to extract images of size [value] from the image to be registered. The second feature block was extracted from the image to be registered. There are two second feature blocks, among which... The initial set of points in the image to be registered is the number of points, and , The number of search points included for each feature point.

4. The method for cross-modal registration of three-dimensional biological images based on domain generalization representation learning according to claim 1, characterized in that: In step (3), the trained deep neural network is obtained through the following steps: (3a1) Obtain whole-brain images of mice with 25µm resolution for 30 fMOST modal images, 25µm resolution for 30 VISoR modal images, and 25µm resolution for 30 STPT modal images. Use the mBrainAligner algorithm to register the fMOST modal mouse brains to the fMOST mouse brain mean template, the VISoR modal mouse brains to the VISoR mouse brain mean template, and the STPT modal mouse brains to the STPT mouse brain mean template. Then, perform semi-automatic optimization on all mouse brains. Register the fMOST mouse brain mean template and the VISoR mouse brain mean template to the STPT mouse brain mean template. (3a2) Take 3700 points in the main texture and contour of the STPT mouse brain mean template, and then use the STPS algorithm to inversely transform these points to the mouse brain of each modality, so that the mouse brain structural information of the same numbered points on each mouse brain of each modality corresponds. (3a3) Using each point on the mouse brain as the center, extract the size on the mouse brain. The image patches are used as the dataset, and 80% of all data in the dataset are randomly selected as the training set and 20% as the validation set. (3a4) Construct a deep neural network, the deep neural network comprising: The feature extraction module is used to extract features from the input first feature block and second feature block to obtain the first target feature map and the first feature map to be registered. The domain generalization module is used to extract features from the input first target feature map and first registration feature map to obtain the second target feature map and the second registration feature map. The metric module is used to measure and calculate the L1 distance between the input second target feature map and the second feature map to be registered, and to obtain the similarity score between the first feature block and the second feature block. (3a5) During training, two image blocks are randomly selected from the training set as the first feature block and the second feature block. Then, data augmentation is performed on the first feature block and the second feature block, and they are fed into the deep neural network for similarity calculation.

5. The method for cross-modal registration of three-dimensional biological images based on domain generalization representation learning according to claim 1, characterized in that: In step (3), calculating the similarity score between the first feature block and the second feature block specifically refers to: extracting the first feature block from the target image The second feature block extracted from the image to be registered The data are sequentially input into a trained deep neural network to predict similarity, resulting in similarity scores. ;in, The size extracted for the i-th point in the target image is First feature block, The size extracted for the j-th search point within the search range of the i-th feature point in the image to be registered is: Second feature block, This represents the similarity score between the i-th point in the target image and the j-th search point within the search range of the i-th point in the image to be registered.

6. The method for cross-modal registration of three-dimensional biological images based on domain generalization representation learning according to claim 1, characterized in that: Step (5) specifically includes the following steps: (5a) The new feature point set sub_new_markers on the image to be registered is obtained from the displacement of the initial point set of the image to be registered. The calculation formula is as follows: ； ； ； in, This represents the x-coordinate of the new i-th feature point on the image to be registered. This represents the y-coordinate of the new i-th feature point on the image to be registered. This represents the z-coordinate of the new i-th feature point on the image to be registered; This represents the x-coordinate of the i-th feature point in the image to be registered before displacement. This represents the y-coordinate of the i-th feature point in the image to be registered before displacement. This represents the z-coordinate of the i-th feature point on the image to be registered before displacement; The displacement of the initial point set of the image to be registered in the x-direction is calculated. The displacement of the initial point set of the image to be registered in the y direction is calculated. The displacement of the initial point set of the image to be registered in the z-direction is calculated. (5b) Draw a new search range from the new feature point set on the image to be registered, and re-infer the similarity between the new feature blocks. That is, return to the iteration of step (2), and replace the initial point set of the image to be registered with the new feature point set on the image to be registered to obtain the second feature block again, until the displacement of the feature points on the image to be registered in step (5). , , If the result is less than 0.1 for 10 consecutive times, the iteration ends and proceeds to step (6).

Images