Continuous pathological image immunohistochemical registration method

Through the continuous pathological image immunohistochemical registration method, fine alignment of adjacent section images is achieved, time-consuming and labor-intensive search of specific cell positions between sections in pathological diagnosis, and diagnostic efficiency and accuracy are improved.

CN119006537APending Publication Date: 2024-11-22SUN YAT SEN UNIV

Patent Information

Application Number
CN202411141267.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In pathological diagnosis, it is necessary to find specific cells at specific locations on multiple continuous paraffin sections. The prior art is time-consuming and labor-intensive, especially for inexperienced doctors, the diagnosis is inefficient.

Method used

Using continuous pathological image immunohistochemical registration method, the immunohistochemical registration alignment at the cell level is achieved by acquiring two adjacent slice images, pre-processing, translation, rotation, affine transformation and iterative registration, and the section image is improved. accuracy.

Benefits of technology

It reduces the work burden of doctors on finding specific cells on different sections, improves diagnostic efficiency and accuracy, and reduces the difficulty of pathologists in the diagnosis of Hodgkin's lymphoma and breast pathology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006537A_ABST
    Figure CN119006537A_ABST
Patent Text Reader

Abstract

The invention provides a continuous pathological image immunohistochemical registration method, which is used for cell-level immunohistochemical registration and alignment of two gigabit-pixel-level tissue pathological sections at adjacent positions. Comprising four parts: image preprocessing, pre-alignment, an affine registration network and a non-rigid registration network. The affine registration network is combined with a converter structure with multi-scale features, and self-learning and interactive learning of cross-scale features are realized by using self-attention and cross-attention mechanisms. This helps to achieve precise structural registration at ultra-high resolution. In addition, the non-rigid registration network solves the deformation problem of the tissue slice. The efficiency and the accuracy of the registration framework are verified through the ANHIR data set and the clinical data set, and high accuracy is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image registration, and more specifically, to a continuous pathological image immunohistochemical registration method. Background Art

[0002] Immunohistochemistry (IHC) is a technique used in medical and biological research that visualizes the presence and distribution of specific proteins or antigens through antibody-antigen interactions, helping pathologists determine the tissue origin of abnormal cells for subsequent differential diagnosis or identification of therapeutic targets.

[0003] In routine diagnosis, pathologists often need to analyze the expression of certain immune markers of specific cells (cells with special morphology or located in a specific location) on the slices. This work needs to be completed through multiple consecutive paraffin sections, and each paraffin section can generally only be stained with 1-2 markers. Pathologists often need to find specific cells in specific locations on different slices and observe their specific performance. This work is challenging, time-consuming and laborious, especially for doctors with less experience and insufficient training.

[0004] For example, in the diagnosis of Hodgkin's lymphoma, the disease is usually manifested as scattered large cells distributed in an inflammatory background, and the immunophenotype of these large cells is very important for diagnosis. Typical Hodgkin cells express CD15 and CD30, but weakly express PAX5 and usually do not express CD20. Since these markers (CD15, CD30, PAX5, CD20) are stained separately on four consecutive sections, pathologists need to accurately identify the same position of the same large cell between sections in order to interpret the IHC results and make a diagnosis. A similar situation exists in the diagnosis of distinguishing between invasive ductal carcinoma and ductal carcinoma in situ in breast pathology, which requires a comprehensive interpretation of epithelial markers (CK) and myoepithelial markers (p63, CK5 / 6) located in the same position on three consecutive sections. Summary of the invention

[0005] The present invention provides a continuous pathological image immunohistochemical registration method, which solves the technical problem in the prior art that it is time-consuming and laborious to search for specific cells at specific positions on different slices.

[0006] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0007] The present invention provides a method for immunohistochemical registration of continuous pathological images, comprising the following steps:

[0008] Obtain two adjacent slice images, one of which is a fixed image and the other is a moving image;

[0009] Preprocessing the fixed image and the moving image to obtain a first tissue image in the fixed image and a second tissue image in the moving image respectively;

[0010] According to the first tissue image and the second tissue image, translating and rotating the second tissue image to obtain a third tissue image;

[0011] According to the first tissue image and the third tissue image, predicting the image affine transformation matrix through a preset multi-scale feature transformer affine transformation network;

[0012] Obtaining a fourth tissue image according to the image affine transformation matrix and the third tissue image;

[0013] Predicting a deformation field of the fourth tissue image using a patch-based iterative registration network according to the first tissue image and the fourth tissue image;

[0014] A fifth tissue image registered with the fixed image is obtained according to the deformation field of the fourth tissue image and the fourth tissue image.

[0015] In the above technical means, two tissue pathology section images at adjacent positions are subjected to cellular-level immunohistochemical registration and alignment. Through preprocessing, the tissue area in the section image is partially segmented, and the subsequent registration is targeted at the segmented tissue area; image pairs with significant misalignment and deflection are quickly calibrated through translation and rotation to improve the accuracy of subsequent registration; images are affine registered through a preset multi-scale feature transformer affine transformation network to further improve image similarity; histological images with local nonlinear deformation are accurately aligned through an iterative registration network to achieve cellular-level immunohistochemical registration and alignment of two tissue pathology section images at adjacent positions.

[0016] Further, the fixed image and the moving image are preprocessed, including:

[0017] Converting the fixed image and the moving image into grayscale images;

[0018] Performing Gaussian filtering to reduce noise on the grayscale image of the fixed image and the grayscale image of the moving image respectively;

[0019] The first U-Net network is used to downsample, fill and segment the grayscale image of the fixed image and the grayscale image of the moving image after denoising to obtain a first tissue image and a second tissue image.

[0020] Further, translating the second tissue image includes:

[0021] respectively calculating the centroids of the first tissue image and the second tissue image;

[0022] Obtaining a translation vector by subtracting the centroid coordinates of the second tissue image from the centroid coordinates of the first tissue image;

[0023] The second tissue image is translated using the translation vector.

[0024] Furthermore, the centroid is calculated as follows:

[0025]

[0026] In the formula, x represents the x-axis coordinate of the center of mass, y represents the y-axis coordinate of the center of mass, and p i represents the gray value of the i-th pixel, x i Indicates the x-axis coordinate of the i-th pixel, y i represents the y-axis coordinate of the i-th pixel, and n represents the total number of pixels.

[0027] Further, rotating the translated second tissue image includes:

[0028] The rotation matrix R(θ) is searched by an exhaustive search strategy and is defined as:

[0029]

[0030] Where θ represents the rotation radian;

[0031] Each searched rotation matrix is ​​multiplied by the translated second tissue image to obtain a rotated image:

[0032]

[0033] In the formula, x′ i Indicates the x-axis coordinate of the i-th pixel after rotation, y i ′ represents the y-axis coordinate of the i-th pixel after rotation;

[0034] Calculate the image intensity similarity index of each rotated image respectively;

[0035] Count the image intensity similarity indexes corresponding to all rotation matrices, and select the rotation matrix with the highest image intensity similarity index as the final rotation matrix;

[0036] The final rotation matrix is ​​multiplied by the translated second tissue image to obtain a third tissue image.

[0037] Furthermore, the image intensity similarity index The calculation method is as follows:

[0038]

[0039] In the formula, Ω represents all pixels, φ represents the deformation field, φ(I M (i)) = R(θ)I M (i) I M represents the second tissue image after translation, I F represents the first tissue image, I F (i) represents the pixel value of the i-th pixel in the first tissue image, I M (i) represents the pixel value of the i-th pixel point in the second tissue image after translation, represents the average pixel value of the first tissue image, I M Represents the average pixel value of the second tissue image after translation.

[0040] Furthermore, the image affine transformation matrix is ​​predicted by a preset multi-scale feature transformer affine transformation network, including:

[0041] The preset multi-scale feature transformer affine transformation network includes a feature pyramid network and a Transformer module;

[0042] Using the feature pyramid network to extract features from the first tissue image and the third tissue image respectively, to obtain a first feature map and a second feature map;

[0043] The first feature map and the second feature map are flattened into a one-dimensional vector, divided into multiple segments, and added with position encoding before being input into the Transformer module to obtain the image affine transformation matrix.

[0044] Furthermore, a patch-based iterative registration network is used to predict the deformation field of the fourth tissue image, including:

[0045] Perform overlapping block processing on the first tissue image and the fourth tissue image to obtain small image blocks of the first tissue image and small image blocks of the fourth tissue image respectively;

[0046] Forming a pair of small image blocks by correspondingly placing the small image blocks of the first tissue image and the small image blocks of the fourth tissue image;

[0047] Inputting the small image block pair into a second U-Net network to predict the deformation field of the small image block;

[0048] After the deformation fields of all the small image blocks are spliced ​​together, the deformation field of the fourth tissue image is obtained.

[0049] Furthermore, the first tissue image and the fourth tissue image are divided into overlapping blocks, wherein the length and width of the overlapping portion are equal to half of the length and width of the small image block.

[0050] Furthermore, after the deformation fields of all small image blocks are spliced ​​together, the following iterative steps are also included:

[0051] In each iteration, the image after the splicing of the deformation field output from the previous iteration is used as input, and after overlapping block processing, it is input into the second U-Net network to obtain the deformation field of this iteration.

[0052] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0053] The present invention proposes a continuous pathological image immunohistochemical registration method, which is used for the fine alignment of two adjacent pathological slices, so that the positions of specific cells at specific positions on different slices are the same, and there is no need to search for specific cells at specific positions on different slices, thereby reducing the workload of doctors. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic diagram of a process of a continuous pathological image immunohistochemical registration method provided by an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of the structure of an affine transformation network of a multi-scale feature transformer provided in an embodiment of the present invention;

[0056] Figure 3 A schematic diagram of a second U-NET network structure provided in an embodiment of the present invention;

[0057] Figure 4 A schematic diagram of a non-rigid registration framework provided by an embodiment of the present invention;

[0058] Figure 5 A schematic diagram of a moving image and a marking point provided by an embodiment of the present invention;

[0059] Figure 6 A schematic diagram of a fixed image and marking points provided by an embodiment of the present invention;

[0060] Figure 7 A schematic diagram of a fixed image and marking points after affine alignment provided by an embodiment of the present invention;

[0061] Figure 8 A schematic diagram of a fixed image and marker points aligned by an iterative registration network provided in an embodiment of the present invention;

[0062] Fig. 9 Another schematic diagram of a moving image and marking points provided by an embodiment of the present invention;

[0063] Fig.10 Another schematic diagram of a fixed image and marking points provided by an embodiment of the present invention;

[0064] Fig.11 A schematic diagram of another fixed image and marking points after affine alignment provided by an embodiment of the present invention;

[0065] Fig.12 Another schematic diagram of a fixed image and marker points aligned by an iterative registration network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0067] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0068] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0069] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0070] Example

[0071] The embodiment of the present invention provides a method for immunohistochemical registration of continuous pathological images. Figure 1 As shown, the following steps are included:

[0072] Obtain two adjacent slice images, one of which is a fixed image and the other is a moving image;

[0073] Preprocessing the fixed image and the moving image to obtain a first tissue image in the fixed image and a second tissue image in the moving image respectively;

[0074] According to the first tissue image and the second tissue image, translating and rotating the second tissue image to obtain a third tissue image;

[0075] According to the first tissue image and the third tissue image, predicting the image affine transformation matrix through a preset multi-scale feature transformer affine transformation network;

[0076] Obtaining a fourth tissue image according to the image affine transformation matrix and the third tissue image;

[0077] Predicting a deformation field of the fourth tissue image using a patch-based iterative registration network according to the first tissue image and the fourth tissue image;

[0078] A fifth tissue image registered with the fixed image is obtained according to the deformation field of the fourth tissue image and the fourth tissue image.

[0079] In the above technical means, two tissue pathology section images at adjacent positions are subjected to cellular-level immunohistochemical registration and alignment. Through preprocessing, the tissue area in the section image is partially segmented, and the subsequent registration is targeted at the segmented tissue area; image pairs with significant misalignment and deflection are quickly calibrated through translation and rotation to improve the accuracy of subsequent registration; images are affine registered through a preset multi-scale feature transformer affine transformation network to further improve image similarity; histological images with local nonlinear deformation are accurately aligned through an iterative registration network to achieve cellular-level immunohistochemical registration and alignment of two tissue pathology section images at adjacent positions.

[0080] In a further embodiment, the fixed image and the moving image are preprocessed, including:

[0081] Converting the fixed image and the moving image into grayscale images;

[0082] Performing Gaussian filtering to reduce noise on the grayscale image of the fixed image and the grayscale image of the moving image respectively;

[0083] The first U-Net network is used to downsample, fill and segment the grayscale image of the fixed image and the grayscale image of the moving image after denoising to obtain a first tissue image and a second tissue image.

[0084] In a further embodiment, translating the second tissue image comprises:

[0085] respectively calculating the centroids of the first tissue image and the second tissue image;

[0086] Obtaining a translation vector by subtracting the centroid coordinates of the second tissue image from the centroid coordinates of the first tissue image;

[0087] The second tissue image is translated using the translation vector so that the coordinates of the center of mass of the second tissue image are in the same position as the coordinates of the center of mass of the first tissue image in the coordinate system.

[0088] In a further embodiment, the centroid is calculated as follows:

[0089]

[0090] In the formula, x represents the x-axis coordinate of the center of mass, y represents the y-axis coordinate of the center of mass, and p i represents the gray value of the i-th pixel, x i Indicates the x-axis coordinate of the i-th pixel, y i represents the y-axis coordinate of the i-th pixel, and n represents the total number of pixels.

[0091] In a further embodiment, rotating the translated second tissue image comprises:

[0092] The rotation matrix R(θ) is searched by an exhaustive search strategy and is defined as:

[0093]

[0094] Where θ represents the rotation radian;

[0095] Each searched rotation matrix is ​​multiplied by the translated second tissue image to obtain a rotated image:

[0096]

[0097] In the formula, x′ i Indicates the x-axis coordinate of the i-th pixel after rotation, y i ′ represents the y-axis coordinate of the i-th pixel after rotation;

[0098] Calculate the image intensity similarity index of each rotated image respectively;

[0099] Count the image intensity similarity indexes corresponding to all rotation matrices, and select the rotation matrix with the highest image intensity similarity index as the final rotation matrix;

[0100] The final rotation matrix is ​​multiplied by the translated second tissue image to obtain a third tissue image.

[0101] In a further embodiment, the image intensity similarity index The calculation method is as follows:

[0102]

[0103] In the formula, Ω represents all pixels, φ represents the deformation field, φ(I M (i)) = R(θ)I M (i) I M represents the second tissue image after translation, I F represents the first tissue image, I F (i) represents the pixel value of the i-th pixel in the first tissue image, I M (i) represents the pixel value of the i-th pixel point in the second tissue image after translation, represents the average pixel value of the first tissue image, I M Represents the average pixel value of the second tissue image after translation.

[0104] In a further embodiment, predicting the image affine transformation matrix by a preset multi-scale feature transformer affine transformation network includes:

[0105] The preset multi-scale feature transformer affine transformation network includes a feature pyramid network (FPN) and a Transformer module;

[0106] Using the feature pyramid network to extract features from the first tissue image and the third tissue image respectively, to obtain a first feature map and a second feature map;

[0107] In this embodiment, the pyramid structure of FPN can extract low-level image features and high-level semantic features at the same time, which is very suitable for the structure of pathological images;

[0108] The first feature map and the second feature map are flattened into a one-dimensional vector, divided into multiple segments, and added with positional encoding before being input into the Transformer module to obtain the image affine transformation matrix.

[0109] In this embodiment, the Transformer module creatively uses the self-attention mechanism and the cross-attention mechanism to make the model pay more attention to the important features of the feature map itself and between feature maps, so as to obtain more detail information that needs to be paid attention to and suppress other useless information. Figure 2 Self-Attention means that input 1 and input 2 are the same, both are fixed images or moving images; while Cross-Attention means that the input feature maps are different, input 1 is a fixed image or a moving image, and input 2 is a moving image or a fixed image.

[0110] In a specific embodiment, the loss function of the preset multi-scale feature transformer affine transformation network is an image intensity similarity index, the loss function is calculated based on the input and output image intensity similarity indexes, and the network is iteratively trained until the network converges.

[0111] In a further embodiment, predicting the deformation field of the fourth tissue image using a patch-based iterative registration network includes:

[0112] The first tissue image and the fourth tissue image are subjected to overlapping block processing to obtain small image blocks of the first tissue image and small image blocks of the fourth tissue image respectively; the overlapping block processing step alleviates the limitation of computer storage and enables registration to be performed at the cellular level;

[0113] Forming a pair of small image blocks by correspondingly placing the small image blocks of the first tissue image and the small image blocks of the fourth tissue image;

[0114] Inputting the small image block pair into a second U-Net network to predict the deformation field of the small image block;

[0115] After the deformation fields of all the small image blocks are spliced ​​together, the deformation field of the fourth tissue image is obtained.

[0116] In a specific embodiment, the second U-Net network is continuously optimized by gradient descent and back propagation algorithms, such as Figure 3 As shown, the second U-Net network uses a lightweight and efficient U-Net, and the non-rigid registration framework can effectively handle patch-based learning and inference. This method allows accurate alignment of histological images with local nonlinear deformations while maintaining computational efficiency. The network uses two-dimensional convolution in both the encoder and decoder stages, using a kernel size of 3 and a stride of 2. After each convolution, a ReLU layer is added to capture the hierarchical features necessary to estimate the final deformation field. By alternating upsampling, convolutional layers, and skip connections, the decoder gradually reconstructs the feature map to match the size of the input image.

[0117] In a further embodiment, the first tissue image and the fourth tissue image are divided into overlapping blocks, wherein the length and width of the overlapping portion are equal to half of the length and width of the small image block.

[0118] In a further embodiment, after the deformation fields of all small image blocks are spliced, the following iterative steps are further included:

[0119] In each iteration, the image after the splicing of the deformation field output from the previous iteration is used as input, and after overlapping block processing, it is input into the second U-Net network to obtain the deformation field of this iteration.

[0120] In a specific embodiment, each iteration predicts a non-rigid deformation field through a patch-based registration framework and acts on the moving image to obtain the input image for the next iteration, such as Figure 4 As shown in the figure. The moving image undergoes continuous deformation, and the final deformed image can be regarded as a combination of multiple local, small displacement deformations. This method enables different iterative layer networks to focus on different areas, and multiple iterative networks work together to improve the accuracy of non-rigid registration of pathological slices. At the same time, the multi-layer iterative method also solves the problem of limited maximum deformation of local areas when the patch size and step size are small.

[0121] In a specific embodiment, the proposed method was tested on the ANHIR public dataset and a unique Hodgkin's lymphoma clinical dataset.

[0122] The ANHIR dataset has 355 different slice images, divided into 251 pairs of evaluation images and 230 pairs of training images. Each slice has 86 manually labeled key points and a size of 120G. The Hodgkin's lymphoma clinical dataset consists of Hodgkin's lymphoma tissue slices, a total of 32 consecutive slices, and 4 pairs of markers are manually labeled, specifically:

[0123] The WSI used is a frozen pathological image from the ANHIR public dataset. The method of the present invention is used to achieve fine registration, specifically:

[0124] S1: read moving images and fixed images and perform preprocessing;

[0125] S2: Perform a quick preliminary alignment of the moving image and the fixed image to eliminate significant misalignment and deflection of the image pair, such as Figure 5 Shown and Figure 6 As shown;

[0126] S3: Affine align the initially aligned moving image and the fixed image, such as Figure 7 As shown;

[0127] S4: Process the high-resolution image into overlapping patches and perform iterative registration at the patch level. After repeated iterations, the non-rigid registration is completed, such as Figure 8 shown.

[0128] The WSI used is Hodgkin's lymphoma tissue slice data. The method of the present invention is used to achieve the immunohistochemical fine registration of continuous pathological images, specifically:

[0129] S1: read moving images and fixed images and perform preprocessing;

[0130] S2: Perform a quick preliminary alignment of the moving image and the fixed image to eliminate significant misalignment and deflection of the image pair, such as Fig. 9 and Fig.10 As shown;

[0131] S3: Affine align the initially aligned moving image and the fixed image, such as Fig.11 As shown;

[0132] S4: Process the high-resolution image into overlapping patches and perform iterative registration at the patch level. After repeated iterations, the non-rigid registration is completed, such as Fig.12 shown.

[0133] Each pair of evaluation images (i, j) has a corresponding marker and l∈L mis the set of marker points of image m. The present invention uses the relative target registration error (rTRE) between the corresponding marker points of two images as a metric to measure the Euclidean distance between the marker points of the mobile image after registration and the real marker points:

[0134]

[0135] There are several different rTRE-based metrics at the marker level, such as median rTRE (MrTRE), mean rTRE (ArTRE), and maximum rTRE (MxrTRE), as well as aggregations of the above metrics and the median, mean, and maximum values.

[0136] In addition, the registration time (including the total time used for data loading, preprocessing and registration) and robustness (the proportion of rTRE reduction after registering all markers of the moving image) are also taken into consideration in the evaluation indicators. Table 1 shows a performance comparison table of the method of the embodiment of the present invention and various algorithms in the same field. Table 1 shows a performance comparison table of the method of the embodiment of the present invention and advanced algorithms in the same field.

[0137] Table 1

[0138]

[0139] Table 2

[0140]

[0141] After comparison, the method of the present invention is comparable to the best method currently available on the ANHIR public data set; and the method of the present invention can also achieve good results on the unique clinical data set. Therefore, the present invention has certain value in clinical application and can assist doctors in clinical diagnosis.

[0142] The same or similar reference numerals correspond to the same or similar components;

[0143] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0144] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for immunohistochemical registration of continuous pathological images, characterized in that: The following steps are involved: Obtain two adjacent slice images, one of which is a fixed image and the other is a moving image; Preprocessing the fixed image and the moving image to obtain a first tissue image in the fixed image and a second tissue image in the moving image respectively; According to the first tissue image and the second tissue image, translating and rotating the second tissue image to obtain a third tissue image; According to the first tissue image and the third tissue image, predicting the image affine transformation matrix through a preset multi-scale feature transformer affine transformation network; Obtaining a fourth tissue image according to the image affine transformation matrix and the third tissue image; Predicting a deformation field of the fourth tissue image using a patch-based iterative registration network according to the first tissue image and the fourth tissue image; A fifth tissue image registered with the fixed image is obtained according to the deformation field of the fourth tissue image and the fourth tissue image.

2. The continuous pathological image immunohistochemical registration method according to claim 1, characterized in that: Preprocessing the fixed image and the moving image includes: Converting the fixed image and the moving image into grayscale images; Performing Gaussian filtering to reduce noise on the grayscale image of the fixed image and the grayscale image of the moving image respectively; The first U-Net network is used to downsample, fill and segment the grayscale image of the fixed image and the grayscale image of the moving image after denoising to obtain a first tissue image and a second tissue image.

3. The immunohistochemical registration method for continuous pathological images according to claim 1, characterized in that: The step of translating the second tissue image includes: respectively calculating the centroids of the first tissue image and the second tissue image; Obtaining a translation vector by subtracting the centroid coordinates of the second tissue image from the centroid coordinates of the first tissue image; The second tissue image is translated using the translation vector.

4. The method for immunohistochemical registration of continuous pathological images according to claim 3, characterized in that: The centroid is calculated as follows: In the formula, x represents the x-axis coordinate of the center of mass, y represents the y-axis coordinate of the center of mass, and p i Represents the gray value of the i-th pixel, x i Indicates the x-axis coordinate of the i-th pixel, y i represents the y-axis coordinate of the i-th pixel, and n represents the total number of pixels.

5. The method for immunohistochemical registration of continuous pathological images according to claim 3, characterized in that: The translated second tissue image is rotated, including: The rotation matrix R(θ) is searched by an exhaustive search strategy and is defined as: Where θ represents the rotation radian; Each searched rotation matrix is ​​multiplied by the translated second tissue image to obtain a rotated image: In the formula, x′ i Represents the x-axis coordinate of the i-th pixel after rotation, y′ i Represents the y-axis coordinate of the i-th pixel after rotation; Calculate the image intensity similarity index of each rotated image respectively; Count the image intensity similarity indexes corresponding to all rotation matrices, and select the rotation matrix with the highest image intensity similarity index as the final rotation matrix; The final rotation matrix is ​​multiplied by the translated second tissue image to obtain a third tissue image.

6. The method for immunohistochemical registration of continuous pathological images according to claim 5, characterized in that: The image intensity similarity index The calculation method is as follows: In the formula, Ω represents all pixels, φ represents the deformation field, φ(I M (i))=() M (i) I M represents the second tissue image after translation, I F represents the first tissue image, I F (i) represents the pixel value of the i-th pixel in the first tissue image, I M (i) represents the pixel value of the i-th pixel point in the second tissue image after translation, represents the average pixel value of the first tissue image, I M Represents the average pixel value of the second tissue image after translation.

7. The continuous pathological image immunohistochemical registration method according to claim 1, characterized in that: The image affine transformation matrix is ​​predicted through a preset multi-scale feature transformer affine transformation network, including: The preset multi-scale feature transformer affine transformation network includes a feature pyramid network and a Transformer module; Using the feature pyramid network to extract features from the first tissue image and the third tissue image respectively, to obtain a first feature map and a second feature map; The first feature map and the second feature map are flattened into a one-dimensional vector, divided into multiple segments, and added with position encoding before being input into the Transformer module to obtain the image affine transformation matrix.

8. The continuous pathological image immunohistochemical registration method according to claim 1, characterized in that: The deformation field of the fourth tissue image is predicted using a patch-based iterative registration network, including: Perform overlapping block processing on the first tissue image and the fourth tissue image to obtain small image blocks of the first tissue image and small image blocks of the fourth tissue image respectively; Forming a pair of small image blocks by correspondingly placing the small image blocks of the first tissue image and the small image blocks of the fourth tissue image; Inputting the small image block pair into a second U-Net network to predict the deformation field of the small image block; After the deformation fields of all the small image blocks are spliced ​​together, the deformation field of the fourth tissue image is obtained.

9. The method for immunohistochemical registration of continuous pathological images according to claim 6, characterized in that: The first tissue image and the fourth tissue image are divided into overlapping blocks, wherein the length and width of the overlapping portion are equal to half of the length and width of the small image block.

10. The method for immunohistochemical registration of continuous pathological images according to claim 6, characterized in that: After splicing the deformation fields of all small image blocks, the following iterative steps are also included: In each iteration, the image after the splicing of the deformation field output from the previous iteration is used as input, and after overlapping block processing, it is input into the second U-Net network to obtain the deformation field of this iteration.

Citation Information

Patent Citations

  • Image registration method based on Swin Transform and CNN double-branch coupling

    CN115082293A

  • Unsupervised deformable three-dimensional medical image registration method and device

    CN118314175A

  • Breast image registration method based on deep learning

    WO2022193750A1

Cited By

  • Method and device for determining transient flow pressure state of pipe explosion of pressure pipe network and medium

    CN120257597A