Unsupervised multi-stained pathology image registration method and system based on optical flow estimation

Abstract

The application discloses a kind of unsupervised multi-staining pathological image registration method and system based on optical flow estimation, wherein the registration method comprises: S1, original data is collected, and data set is constructed;S2, the feature map of each image in the data set is extracted, to form a feature image set F;S3, the registration network is trained using the feature image set;S4, the fixed image and the floating image of the multi-staining pathological image to be registered are input into the trained registration network, and the output of each branch of the registration network is the multi-staining pathological image after registration.The registration method does not need manual intervention, and can realize the non-rigid registration of pathological images.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically relating to an unsupervised multi-stain pathological image registration method and registration system based on optical flow estimation. Background Technology

[0002] In basic histology research, analyzing continuous but differently stained histological sections allows for precise descriptions of cell morphology, size, and location on pathological sections, and yields valuable conclusions regarding molecular and cellular interactions, tissue growth, and structural relationships. The process of preparing and analyzing tissue sections involves transferring fixed tissue to paraffin blocks and slicing it into 2–5 μm thin sections. These sections are then stained with immunohistochemistry and examined by scientists or physicians using conventional or virtual microscopes. Inevitably, non-rigid deformation occurs in the tissue sections during preparation and chemical staining, posing a significant challenge to subsequent image analysis.

[0003] Currently, most traditional non-rigid multi-stain pathological image registration methods and deep learning-based methods require manually labeled key point information in the registration data. Obtaining such data is a very laborious and subjective process because it usually requires manual operation and expert participation. Summary of the Invention

[0004] Purpose of the invention: To address the problems existing in the prior art, the present invention provides an unsupervised multi-stain pathological image registration method based on optical flow estimation. This registration method does not require human intervention and can achieve non-rigid registration of pathological images.

[0005] Technical solution: This invention discloses an unsupervised multi-stain pathological image registration method based on optical flow estimation, comprising the following steps:

[0006] S1. Collect raw data and construct a dataset; the dataset consists of multiple samples, where the first... The sample is , A fixed image of a multi-stained pathological image. yes The corresponding number Zhang floating image, , ; The number of samples in the dataset. The number of floating images corresponding to a fixed image;

[0007] S2. Extract the feature map of each image in the dataset to form the feature image set F:

[0008]

[0009] in yes The corresponding feature map, yes The corresponding feature map;

[0010] S3. Train the registration network using the feature image set, specifically including:

[0011] S31. Construct a registration network and use the registration network to perform deformation calculation on the feature map of the floating image;

[0012] The registration network is composed of The system consists of several parallel branches, each with an identical structure, comprising a pyramid structure and a variable field structure. The pyramid structure is composed of K consecutive downsampling layers, numbered sequentially from 0 to K-1. The variable field structure is composed of K-1 consecutive upsampling variable fields, numbered sequentially from K-1 to K-2, ..., K-1, where layers K-1, K-2, ..., K-2 are double-upsampling variable fields, and layer 1 is a quadruple-upsampling variable field. Layer sampling variable field pair The output of the subsampled layer is used for deformation calculation. ;

[0013] S32. Samples from feature image set F The first in the input registration network The first branch passes through the K-layer downsampling layer, then through the K-1-layer upsampling variable field; Layer sampling variable field pair Deformation calculation is performed on the output of the layer undersampling:

[0014]

[0015] in , It is the first Layer downsampling layer for floating image feature maps Image features after processing; It is the first Layer sampling type variable field, yes The first result obtained after deformation calculation Layered floating image feature map It is a type change operator;

[0016] if ,calculate and The correlation is calculated by the optical flow estimator. Layer upsampling variable field ; It is the first Layer downsampling layer for fixed image feature maps Image features after processing;

[0017] if The output of the first-layer upsampling variable field is a floating image feature map. morphological characteristics ;

[0018] S33, Calculate the first Fixed image feature map of each branch and similarity :

[0019]

[0020] in express The characteristic average value, express The characteristic average value, It involves calculating the mean of the image feature matrix;

[0021] calculate Parameter distance of each branch :

[0022] in These are weighting coefficients. These represent the parameters of each branch; Construct a parameter matrix, The square of the 2-norm of the parameter matrix;

[0023] Calculate the total loss function: ;

[0024] S34. Repeat steps S32 and S33 using samples from the feature image set F, minimizing the total loss function. Value, optimization Train the registration network;

[0025] S4. Input the fixed and floating images of the multi-stain pathological images to be registered into the trained registration network. The output of each branch of the registration network is the registered multi-stain pathological image.

[0026] Further, step S1 specifically includes:

[0027] S11. Collect HE staining images of different histologies, as well as multiple IHC staining images corresponding to each HE staining image, to form the original dataset;

[0028] S12. Convert the RGB images in the original dataset to grayscale images;

[0029] S13. Process the images in the original dataset using the Sobel descriptor to obtain gradient images; the kernel size of the Sobel descriptor is 7; then perform smoothing processing using a median filter, the kernel size of which is 5; the resulting gradient map is then padded, downsampled, and its dimensions are unified to obtain the dataset. ,in This is a fixed image of a multi-stain pathological image, that is, an image obtained after processing the HE-stained image through steps S12 and S13. yes The corresponding number Zhang floating image, i.e. The corresponding number IHC staining images, , ; The number of samples in the dataset. This represents the number of floating images corresponding to a fixed image, i.e., the number of IHC staining images corresponding to a HE staining image.

[0030] Furthermore, step S2 specifically includes:

[0031] S21. Input each image in the dataset into the ResNet network to initially extract feature maps;

[0032] S22. Input the initially extracted feature map into the transformer-based feature encoding module to obtain the encoded features of each image; convert the encoded features into a one-dimensional vector and add positional encoding information to obtain the fused features of each image;

[0033] S23. For each image, perform Self-Attention calculation on its own fused features and Cross-Attention calculation between groups to obtain enhanced features, which constitute the feature image set F corresponding to the dataset.

[0034] Furthermore, K=6, that is: each branch in the registration network includes a pyramid structure and a variable field structure, wherein the pyramid structure consists of 6 consecutive downsampling layers and the variable field structure consists of 5 consecutive upsampling variable fields.

[0035] Furthermore, the number of feature channels in the six consecutive downsampling layers of the pyramid structure are 16, 32, 64, 96, 128, and 196, respectively.

[0036] On the other hand, the present invention also discloses a system for implementing the above-mentioned unsupervised multi-stain pathological image registration method based on optical flow estimation, comprising:

[0037] Dataset construction module 100 is used to construct a dataset based on the original data, the dataset consisting of multiple samples, wherein the first sample is the first sample. The sample is , A fixed image of a multi-stained pathological image. yes The corresponding number Zhang floating image, , ; The number of samples in the dataset. The number of floating images corresponding to a fixed image;

[0038] Feature image set construction module 200 is used to extract feature maps from each image in the dataset to form a feature image set F:

[0039]

[0040] in yes The corresponding feature map, yes The corresponding feature map;

[0041] Registration network 300, the registration network being composed of The system consists of several parallel branches, each with an identical structure, comprising a pyramid structure and a variable field structure. The pyramid structure is composed of K consecutive downsampling layers, numbered sequentially from 0 to K-1. The variable field structure is composed of K-1 consecutive upsampling variable fields, numbered sequentially from K-1 to K-2, ..., K-1, where layers K-1, K-2, ..., K-2 are double-upsampling variable fields, and layer 1 is a quadruple-upsampling variable field. Layer sampling variable field pair The output of the subsampled layer is used for deformation calculation. .

[0042] Furthermore, the training steps of the registration network 300 are as follows:

[0043] S71. Samples from feature image set F The first in the input registration network The first branch passes through the K-layer downsampling layer, then through the K-1-layer upsampling variable field; Layer sampling variable field pair Deformation calculation is performed on the output of the layer undersampling:

[0044]

[0045] in , It is the first Layer downsampling layer for floating image feature maps Image features after processing; It is the first Layer sampling type variable field, yes The first result obtained after deformation calculation Layered floating image feature map It is a type change operator;

[0046] if ,calculate and The correlation is calculated by the optical flow estimator. Layer upsampling variable field ; It is the first Layer downsampling layer for fixed image feature maps Image features after processing;

[0047] if The output of the first-layer upsampling variable field is a floating image feature map. morphological characteristics ;

[0048] S72, Calculate the first Fixed image feature map of each branch and similarity :

[0049]

[0050] in express The characteristic average value, express The characteristic average value, It involves calculating the mean of the image feature matrix;

[0051] calculate Parameter distance of each branch :

[0052] in These are weighting coefficients. These represent the parameters of each branch; Construct a parameter matrix, The square of the 2-norm of the parameter matrix;

[0053] Calculate the total loss function: ;

[0054] S73. Repeat steps S71 and S72 using samples from the feature image set F, minimizing the total loss function. Value, optimization The registration network is trained.

[0055] Furthermore, the specific steps for the dataset construction module 100 to construct the dataset are as follows:

[0056] S11. Collect HE staining images of different histologies, as well as multiple IHC staining images corresponding to each HE staining image, to form the original dataset;

[0057] S12. Convert the RGB images in the original dataset to grayscale images;

[0058] S13. Process the images in the original dataset using the Sobel descriptor to obtain gradient images; the kernel size of the Sobel descriptor is 7; then perform smoothing processing using a median filter, the kernel size of which is 5; the resulting gradient map is then padded, downsampled, and its dimensions are unified to obtain the dataset. ,in This is a fixed image of a multi-stain pathological image, that is, an image obtained after processing the HE-stained image through steps S12 and S13. yes The corresponding number Zhang floating image, i.e. The corresponding number IHC staining images, , ; The number of samples in the dataset. This represents the number of floating images corresponding to a fixed image, i.e., the number of IHC staining images corresponding to a HE staining image.

[0059] Furthermore, the specific steps for the feature image set construction module 200 to construct the feature image set are as follows:

[0060] S21. Input each image in the dataset into the ResNet network to initially extract feature maps;

[0061] S22. Input the initially extracted feature map into the transformer-based feature encoding module to obtain the encoded features of each image; convert the encoded features into a one-dimensional vector and add positional encoding information to obtain the fused features of each image;

[0062] S23. For each image, perform Self-Attention calculation on its own fused features and Cross-Attention calculation between groups to obtain enhanced features, which constitute the feature image set F corresponding to the dataset.

[0063] Furthermore, K=6, that is: each branch in the registration network includes a pyramid structure and a variable field structure, wherein the pyramid structure consists of 6 consecutive downsampling layers and the variable field structure consists of 5 consecutive upsampling variable fields.

[0064] Beneficial effects: The unsupervised multi-stain pathological image registration method and system based on optical flow estimation disclosed in this invention have the following advantages:

[0065] 1. The registration method and registration system disclosed in this invention do not require manual annotation and are capable of high-precision pathological image registration in non-rigid deformation scenarios, thus solving the dilemma of manual calibration in the current pathological image analysis process;

[0066] 2. In practical applications, this invention can effectively assist users in analyzing multi-staining pathological images to complete downstream tasks such as combined analysis of gene expression profiles of multiple specimens, disease grading, and classification. Attached Figure Description

[0067] Figure 1 This is a flowchart of the unsupervised multi-stain pathological image registration method based on optical flow estimation disclosed in this invention;

[0068] Figure 2 A schematic diagram of the registration network structure;

[0069] Figure 3 A schematic diagram of the structure of a branch in a registration network;

[0070] Figure 4 This is a schematic diagram of the structure of the unsupervised multi-stain pathological image registration system based on optical flow estimation disclosed in this invention. Detailed Implementation

[0071] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0072] This invention discloses an unsupervised multi-stain pathological image registration method based on optical flow estimation, such as... Figure 1 As shown, it includes the following steps:

[0073] S1. Collect raw data and construct a dataset; the dataset consists of multiple samples, where the first... The sample is , A fixed image of a multi-stained pathological image. yes The corresponding number Zhang floating image, , ; The number of samples in the dataset. This refers to the number of floating images corresponding to a fixed image; specifically:

[0074] S11. Collect HE staining images of different histologies, as well as multiple IHC staining images corresponding to each HE staining image, to form the original dataset;

[0075] S12. Convert the RGB images in the original dataset to grayscale images;

[0076] S13. The Sobel descriptor is used to process the images in the original dataset to obtain gradient images. To preserve detailed structural information, such as cell positions, this embodiment uses a Sobel descriptor with a kernel size of 7 to obtain the gradient images. A median filter with a kernel size of 5 is then used for smoothing. The resulting gradient map is then padded, downsampled, and its dimensions are standardized to obtain the dataset. ,in This is a fixed image of a multi-stain pathological image, that is, an image obtained after processing the HE-stained image through steps S12 and S13. yes The corresponding number Zhang floating image, i.e. The corresponding number IHC staining images;

[0077] S2. Extract the feature map of each image in the dataset to form the feature image set F:

[0078]

[0079] in yes The corresponding feature map, yes The corresponding feature map;

[0080] Step S2 specifically includes:

[0081] S21. Input each image in the dataset into the ResNet network to initially extract feature maps;

[0082] S22. Input the initially extracted feature map into the transformer-based feature encoding module to obtain the encoded features of each image; convert the encoded features into a one-dimensional vector and add positional encoding information to obtain the fused features of each image;

[0083] S23. For each image, perform Self-Attention calculation on its own fused features and Cross-Attention calculation between groups to obtain enhanced features, which constitute the feature image set F corresponding to the dataset.

[0084] The feature map extraction of the image mainly uses a transformer-based feature encoding module. For detailed steps, please refer to: Liu Z, Lin Y, Cao Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision. 2021: 10012-10022.

[0085] S3. Train the registration network using the feature image set, specifically including:

[0086] S31. Construct a registration network and use the registration network to perform deformation calculation on the feature map of the floating image;

[0087] like Figure 2 As shown, the registration network consists of The system consists of several parallel branches, each with the same structure, consisting of a pyramid structure and a variable field structure. The pyramid structure is composed of K consecutive downsampling layers, numbered sequentially as 0, 1, 2, ..., K-1. The variable field structure is composed of K-1 consecutive upsampling variable fields, numbered sequentially as K-1, K-2, ..., 2, 1.

[0088] In this embodiment, K=6, meaning that each branch in the registration network includes a pyramid structure and a deformation field structure. The pyramid structure consists of six consecutive downsampling layers, numbered sequentially as 0. The 5-type variable field structure consists of five consecutive layers of upsampled variable fields, numbered sequentially as 5. 1. No. The structure of each branch is as follows: Figure 3 As shown in the figure, the rectangles represent downsampling layers in the pyramid structure, implementing downsampling regularization; the numbers in the rectangles represent the numbering. The number of feature channels in six consecutive downsampling layers in the pyramid structure are 16, 32, 64, 96, 128, and 196, respectively, meaning that the larger the layer number, the larger the number of feature channels. In a 5-layer upsampling variable field, 5 Layer 2 is a double-upsampled variable field, and Layer 1 is a quadruple-upsampled variable field; Layer sampling variable field pair The output of the subsampled layer is used for deformation calculation. ;

[0089] S32. Samples from feature image set F The first in the input registration network The first branch passes through the K-layer downsampling layer, then through the K-1-layer upsampling variable field; Layer sampling variable field pair Deformation calculation is performed on the output of the layer undersampling:

[0090]

[0091] in , It is the first Layer downsampling layer for floating image feature maps Image features after processing; It is the first Layer sampling type variable field, yes The first result obtained after deformation calculation Layered floating image feature map It is a type change operator;

[0092] if ,calculate and The correlation is calculated by the optical flow estimator. Layer upsampling variable field ; It is the first Layer downsampling layer for fixed image feature maps Image features after processing;

[0093] The optical flow estimator calculates the new upsampled variable field using the method described in the literature: Sun D, ​​Yang X, Liu MY, et al. Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 8934-8943.

[0094] if The output of the first-layer upsampling variable field is a floating image feature map. morphological characteristics ;

[0095] S33, Calculate the first Fixed image feature map of each branch and similarity :

[0096]

[0097] in express The characteristic average value, express The characteristic average value, It involves calculating the mean of the image feature matrix;

[0098] calculate Parameter distance of each branch :

[0099] in These are weighting coefficients. These represent the parameters of each branch; Construct a parameter matrix, The square of the 2-norm of the parameter matrix;

[0100] Calculate the total loss function: ;

[0101] S34. Repeat steps S32 and S33 using samples from the feature image set F, minimizing the total loss function. Value, optimization Train the registration network;

[0102] In this embodiment, backpropagation is used to update the parameters of each branch of the configuration network.

[0103] S4. Input the fixed and floating images of the multi-stain pathological images to be registered into the trained registration network. The output of each branch of the registration network is the registered multi-stain pathological image.

[0104] This invention also discloses a system for implementing the above-mentioned unsupervised multi-stain pathological image registration method based on optical flow estimation, such as... Figure 4 As shown, it includes:

[0105] Dataset construction module 100 is used to construct a dataset based on the original data, the dataset consisting of multiple samples, wherein the first sample is the first sample. The sample is , A fixed image of a multi-stained pathological image. yes The corresponding number Zhang floating image, , ; The number of samples in the dataset. The number of floating images corresponding to a fixed image;

[0106] Feature image set construction module 200 is used to extract feature maps from each image in the dataset to form a feature image set F:

[0107]

[0108] in yes The corresponding feature map, yes The corresponding feature map;

[0109] Registration network 300, the registration network being composed of The system consists of several parallel branches, each with an identical structure, comprising a pyramid structure and a variable field structure. The pyramid structure is composed of K consecutive downsampling layers, numbered sequentially from 0 to K-1. The variable field structure is composed of K-1 consecutive upsampling variable fields, numbered sequentially from K-1 to K-2, ..., K-1, where layers K-1, K-2, ..., K-2 are double-upsampling variable fields, and layer 1 is a quadruple-upsampling variable field. Layer sampling variable field pair The output of the subsampled layer is used for deformation calculation. .

[0110] The registration network 300 described above is trained according to steps S32-S34. When registering multi-stained pathological images, a fixed image of the multi-stained pathological image to be registered is used. and corresponding A floating image constitute Image pairs , ;Will Each image is input to the registration network. In each branch, the output of each branch is the same as the fixed image. Registered multi-stain pathological images.

Claims

1. An unsupervised multi-stain pathological image registration method based on optical flow estimation, characterized in that, Includes the following steps: S1. Collect raw data and construct a dataset; the dataset consists of multiple samples, where the first... The sample is , A fixed image of a multi-stained pathological image. yes The corresponding number Zhang floating image, , ; The number of samples in the dataset. The number of floating images corresponding to a fixed image; S2. Extract the feature map of each image in the dataset to form the feature image set F: in yes The corresponding feature map, yes The corresponding feature map; S3. Train the registration network using the feature image set, specifically including: S31. Construct a registration network and use the registration network to perform deformation calculation on the feature map of the floating image; The registration network is composed of The system consists of several parallel branches, each with an identical structure, comprising a pyramid structure and a variable field structure. The pyramid structure is composed of K consecutive downsampling layers, numbered sequentially from 0 to K-1. The variable field structure is composed of K-1 consecutive upsampling variable fields, numbered sequentially from K-1 to K-2, ..., K-1, where layers K-1, K-2, ..., K-2 are double-upsampling variable fields, and layer 1 is a quadruple-upsampling variable field. Layer sampling variable field pair The output of the subsampled layer is used for deformation calculation. ; S32. Samples from feature image set F The first in the input registration network The first branch passes through the K-layer downsampling layer, then through the K-1-layer upsampling variable field; Layer sampling variable field pair Deformation calculation is performed on the output of the layer undersampling: in , It is the first Layer downsampling layer for floating image feature maps Image features after processing; It is the first Layer sampling type variable field, yes The first result obtained after deformation calculation Layered floating image feature map It is a type change operator; if ,calculate and The correlation is calculated by the optical flow estimator. Layer upsampling variable field ; It is the first Layer downsampling layer for fixed image feature maps Image features after processing; if The output of the first layer upsampling variable field is a floating image feature map. morphological characteristics ; S33, Calculate the first Fixed image feature map of each branch and similarity : in express The characteristic average value, express The characteristic average value, It involves calculating the mean of the image feature matrix; calculate Parameter distance of each branch : in These are weighting coefficients. These represent the parameters of each branch; Construct a parameter matrix, The square of the 2-norm of the parameter matrix; Calculate the total loss function: ; S34. Repeat steps S32 and S33 using samples from the feature image set F, minimizing the total loss function. Value, optimization Train the registration network; S4. Input the fixed and floating images of the multi-stain pathological images to be registered into the trained registration network. The output of each branch of the registration network is the registered multi-stain pathological image.

2. The unsupervised multi-stain pathological image registration method based on optical flow estimation according to claim 1, characterized in that, Step S1 specifically involves: S11. Collect HE staining images of different histologies, as well as multiple IHC staining images corresponding to each HE staining image, to form the original dataset; S12. Convert the RGB images in the original dataset to grayscale images; S13. Process the images in the original dataset using the Sobel descriptor to obtain gradient images; the kernel size of the Sobel descriptor is 7; then perform smoothing processing using a median filter, the kernel size of which is 5; the resulting gradient map is then padded, downsampled, and its dimensions are unified to obtain the dataset. ,in This is a fixed image of a multi-stain pathological image, that is, an image obtained after processing the HE-stained image through steps S12 and S13. yes The corresponding number Zhang floating image, i.e. The corresponding number IHC staining images, , ; The number of samples in the dataset. This represents the number of floating images corresponding to a fixed image, i.e., the number of IHC staining images corresponding to a HE staining image.

3. The unsupervised multi-stain pathological image registration method based on optical flow estimation according to claim 1, characterized in that, Step S2 specifically includes: S21. Input each image in the dataset into the ResNet network to initially extract feature maps; S22. Input the initially extracted feature map into the transformer-based feature encoding module to obtain the encoded features of each image; convert the encoded features into a one-dimensional vector and add positional encoding information to obtain the fused features of each image; S23. For each image, perform Self-Attention calculation on its own fused features and Cross-Attention calculation between groups to obtain enhanced features, which constitute the feature image set F corresponding to the dataset.

4. The unsupervised multi-stain pathological image registration method based on optical flow estimation according to claim 1, characterized in that, K=6, meaning that each branch in the registration network includes a pyramid structure and a variable field structure. The pyramid structure consists of 6 consecutive downsampling layers, and the variable field structure consists of 5 consecutive upsampling variable fields.

5. The unsupervised multi-stain pathological image registration method based on optical flow estimation according to claim 4, characterized in that, The number of feature channels in the six consecutive downsampling layers of the pyramid structure are 16, 32, 64, 96, 128, and 196, respectively.

6. An unsupervised multi-stain pathological image registration system based on optical flow estimation, characterized in that, include: Dataset construction module (100) is used to construct a dataset based on the original data, the dataset consisting of multiple samples, wherein the first sample is the first sample. The sample is , A fixed image of a multi-stained pathological image. yes The corresponding number Zhang floating image, , ; The number of samples in the dataset. The number of floating images corresponding to a fixed image; The feature image set construction module (200) is used to extract the feature map of each image in the dataset to form the feature image set F: in yes The corresponding feature map, yes The corresponding feature map; Registration network (300), the registration network being composed of The system consists of several parallel branches, each with an identical structure, comprising a pyramid structure and a variable field structure. The pyramid structure is composed of K consecutive downsampling layers, numbered sequentially from 0 to K-1. The variable field structure is composed of K-1 consecutive upsampling variable fields, numbered sequentially from K-1 to K-2, ..., K-1, where layers K-1, K-2, ..., K-2 are double-upsampling variable fields, and layer 1 is a quadruple-upsampling variable field. Layer sampling variable field pair The output of the subsampled layer is used for deformation calculation. ; The training steps for the registration network (300) are as follows: S71. Samples from feature image set F The first in the input registration network The first branch passes through the K-layer downsampling layer, then through the K-1-layer upsampling variable field; Layer sampling variable field pair Deformation calculation is performed on the output of the layer undersampling: in , It is the first Layer downsampling layer for floating image feature maps Image features after processing; It is the first Layer sampling type variable field, yes The first result obtained after deformation calculation Layered floating image feature map It is a type change operator; if ,calculate and The correlation is calculated by the optical flow estimator. Layer upsampling variable field ; It is the first Layer downsampling layer for fixed image feature maps Image features after processing; if The output of the first layer upsampling variable field is a floating image feature map. morphological characteristics ; S72, Calculate the first Fixed image feature map of each branch and similarity : in express The characteristic average value, express The characteristic average value, This represents the calculation of the mean of the image feature matrix; calculate Parameter distance of each branch : in These are weighting coefficients. These represent the parameters of each branch model; Construct a parameter matrix, The square of the 2-norm of the parameter matrix; Calculate the total loss function: ; S73. Repeat steps S71 and S72 using samples from the feature image set F, minimizing the total loss function. Value, optimization Train the registration network; The registration network is used to perform shape calculation on the feature map of the floating image. The fixed image and the floating image of the multi-stained pathological image to be registered are input into the trained registration network, and the output of each branch of the registration network is the registered multi-stained pathological image.

7. The unsupervised multi-stain pathological image registration system based on optical flow estimation according to claim 6, characterized in that, The specific steps for the dataset construction module (100) to construct the dataset are as follows: S11. Collect HE staining images of different histologies, as well as multiple IHC staining images corresponding to each HE staining image, to form the original dataset; S12. Convert the RGB images in the original dataset to grayscale images; S13. Process the images in the original dataset using the Sobel descriptor to obtain gradient images; the kernel size of the Sobel descriptor is 7; then perform smoothing processing using a median filter, the kernel size of which is 5; the resulting gradient map is then padded, downsampled, and its dimensions are unified to obtain the dataset. ,in This is a fixed image of a multi-stain pathological image, that is, an image obtained after processing the HE-stained image through steps S12 and S13. yes The corresponding number Zhang floating image, i.e. The corresponding number IHC staining images, , ; The number of samples in the dataset. This represents the number of floating images corresponding to a fixed image, i.e., the number of IHC staining images corresponding to a HE staining image.

8. The unsupervised multi-stain pathological image registration system based on optical flow estimation according to claim 6, characterized in that, The specific steps of the feature image set construction module (200) in constructing the feature image set are as follows: S21. Input each image in the dataset into the ResNet network to initially extract feature maps; S22. Input the initially extracted feature map into the transformer-based feature encoding module to obtain the encoded features of each image; convert the encoded features into a one-dimensional vector and add positional encoding information to obtain the fused features of each image; S23. For each image, perform Self-Attention calculation on its own fused features and Cross-Attention calculation between groups to obtain enhanced features, which constitute the feature image set F corresponding to the dataset.

9. The unsupervised multi-stain pathological image registration system based on optical flow estimation according to claim 6, characterized in that, K=6, meaning that each branch in the registration network includes a pyramid structure and a variable field structure. The pyramid structure consists of 6 consecutive downsampling layers, and the variable field structure consists of 5 consecutive upsampling variable fields.

Images