Whole slide pathology image registration method, apparatus, device, and storage medium

Abstract

Embodiments of the present application provide a registration method, device and equipment for whole slide pathological images and a storage medium. Applied to the field of histopathological image registration, the method comprises the following steps: obtaining a whole slide pathological image, and preprocessing the obtained whole slide pathological image; constructing a global affine transformation matrix, performing affine transformation on the preprocessed whole slide pathological image through the global affine transformation matrix, and obtaining a preliminary registration image; constructing a displacement field based on the global affine transformation matrix, optimizing the constructed displacement field, performing displacement transformation on the preliminary registration image according to the optimized displacement field, and obtaining a final registration image. The method reduces the calculation cost and improves the accuracy of the registration result.

Description

Technical Field

[0001] This application relates to the field of histopathological image registration, and more particularly to a registration method, apparatus, device, and storage medium for whole-section pathological images. Background Technology

[0002] Registration, as a fundamental technology in medical image analysis, aims to eliminate spatial and radiometric differences between different images and achieve accurate alignment and fusion of multi-source data.

[0003] Currently, in the field of whole-slide image (WSI) registration, the main methods include traditional feature matching methods, deep learning methods, and hybrid methods. Traditional feature matching methods, such as SIFT and RIFT, perform well in handling scale and rotation invariance, but encounter problems with decreased matching accuracy in images with varying staining intensity and high resolution. Deep learning methods, such as SuperPoint and RegWSI, can improve matching accuracy through end-to-end training, but they rely heavily on large amounts of labeled data and have high computational costs, and it is difficult to maintain efficiency at high resolutions. Hybrid methods combine traditional optimization and deep learning feature extraction, but still perform poorly when the rotation angle is large or the image resolution is too high.

[0004] Existing methods generally suffer from insufficient rotation invariance, sensitivity to changes in illumination and staining, poor adaptability to high-resolution images, and high computational complexity, which limit their effectiveness and efficiency in pathological image registration applications. Summary of the Invention

[0005] This application provides a registration method, apparatus, device, and storage medium for whole-section pathological images to address the problems of insufficient rotation invariance, sensitivity to changes in illumination and staining, poor adaptability to high-resolution images, and high computational complexity in existing methods.

[0006] In a first aspect, this application provides a registration method for whole-section pathological images, the method comprising:

[0007] Obtain whole-section pathological images and preprocess the obtained whole-section pathological images;

[0008] A global affine transformation matrix is ​​constructed, and the preprocessed whole-slice pathological image is subjected to affine transformation using the global affine transformation matrix to obtain a preliminary registration image;

[0009] A displacement field is constructed based on a global affine transformation matrix, and the constructed displacement field is optimized. The displacement transformation is then applied to the preliminary registered image based on the optimized displacement field to obtain the final registered image.

[0010] Optionally, constructing the global affine transformation matrix includes:

[0011] A corner detection algorithm was used to extract multiple feature points from the preprocessed whole-slice pathological image;

[0012] The descriptor for each feature point is determined by a preset frequency domain filter bank;

[0013] Descriptors in the target image and the preprocessed whole-slice pathological image are matched based on similarity to obtain multiple candidate matching pairs;

[0014] The transformation parameters are calculated based on multiple candidate matching pairs using a fast sampling consensus algorithm, and the global affine transformation matrix is ​​determined based on the obtained transformation parameters.

[0015] Optionally, the corner detection algorithm is used to extract multiple feature points from the preprocessed whole-slice pathological image, including:

[0016] For each candidate pixel in the preprocessed whole-slice pathological image, a circular neighborhood centered on the candidate pixel is defined.

[0017] The gray values ​​of candidate pixels within a circular neighborhood are compared with those of other pixels to determine whether the gray value difference between the two points is greater than a preset gray value difference threshold.

[0018] If the number of pixels in the circular neighborhood whose grayscale difference is greater than the preset grayscale difference threshold exceeds the preset threshold, then the candidate pixels are used as feature points.

[0019] Optionally, determining the descriptor for each feature point using a preset frequency domain filter bank includes:

[0020] For each feature point in the preprocessed whole-slice pathological image, a square neighborhood centered on the feature point is defined;

[0021] Use the meshgrid function to generate a relative grid coordinate matrix within a square neighborhood;

[0022] The logarithmic amplitude of the feature point is determined based on the Euclidean distance from each relative coordinate point in the square neighborhood to the center point.

[0023] The angle of the feature point is determined based on the angle of each relative coordinate point relative to the center point;

[0024] The logarithmic amplitude of the feature points is discretized according to a preset radius threshold to obtain a discrete amplitude index;

[0025] Discretize the angles of the feature points to obtain discrete direction indices;

[0026] A multidimensional histogram is constructed based on the discrete amplitude index and the discrete direction index, and the constructed multidimensional histogram is used as the descriptor of the feature point.

[0027] Optionally, after constructing a multidimensional histogram based on the discrete amplitude index and discrete direction index, and using the constructed multidimensional histogram as a descriptor for the feature point, the method further includes:

[0028] Gaussian weighting is applied to the discrete amplitude indices of each point within the square neighborhood, and a weighted direction histogram about the discrete direction indices is constructed based on the Gaussian weighted discrete amplitude indices.

[0029] The weighted orientation histogram is smoothed, and peak detection and interpolation are performed on the smoothed weighted orientation histogram to determine the main orientation of the feature point neighborhood;

[0030] Based on a determined principal direction, the descriptor of the feature points is cyclically shifted to obtain a rotationally normalized descriptor.

[0031] Optionally, the step of constructing a displacement field based on a global affine transformation matrix, optimizing the constructed displacement field, and performing a displacement transformation on the preliminary registered image according to the optimized displacement field to obtain the final registered image includes:

[0032] The initial displacement field is constructed using an affine transformation matrix;

[0033] The initial displacement field is optimized step by step according to the preset optimization objective function to obtain the optimized displacement field. The preset optimization objective function includes: a similarity metric term and a regularization term.

[0034] The optimized displacement field is used to perform displacement transformation on the deformation of the initial registered image to obtain the final registered image.

[0035] Optionally, the preprocessing of the acquired whole-section pathological images includes:

[0036] Convert the acquired whole-section pathological images into grayscale images;

[0037] Perform contrast enhancement processing on a grayscale image to generate an enhanced grayscale image;

[0038] The enhanced grayscale image is downsampled at multiple scales to generate an image pyramid structure with multiple resolution levels.

[0039] Secondly, this application provides a registration device for whole-section pathological images, the device comprising:

[0040] The acquisition module is used to acquire whole-slice pathological images;

[0041] The first processing module is used to preprocess the acquired whole-slice pathological images;

[0042] The second processing module is used to construct a global affine transformation matrix, and to perform affine transformation on the preprocessed whole-slice pathological image through the global affine transformation matrix to obtain a preliminary registration image.

[0043] The second processing module is also used to construct a displacement field based on the global affine transformation matrix, optimize the constructed displacement field, and perform displacement transformation on the preliminary registered image according to the optimized displacement field to obtain the final registered image.

[0044] Thirdly, this application provides a registration device for whole-slice pathological images, comprising:

[0045] Memory;

[0046] processor;

[0047] The memory stores computer-executed instructions;

[0048] The processor executes computer execution instructions stored in the memory to implement the registration method for whole-slice pathological images as described in the first aspect and various possible implementations of the first aspect above.

[0049] Fourthly, this application provides a computer storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the registration method for whole-slice pathological images as described in the first aspect and various possible implementations of the first aspect.

[0050] This application provides a method, apparatus, device, and storage medium for registering whole-slice pathological images. The method involves acquiring a whole-slice pathological image and preprocessing it; constructing a global affine transformation matrix; performing an affine transformation on the preprocessed whole-slice pathological image using the global affine transformation matrix to obtain a preliminary registered image; constructing a displacement field based on the global affine transformation matrix; optimizing the constructed displacement field; and performing a displacement transformation on the preliminary registered image based on the optimized displacement field to obtain the final registered image. This reduces computational costs and improves the accuracy of the registration results. Attached Figure Description

[0051] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0052] Figure 1 A schematic flowchart illustrating the registration method for whole-slice pathological images provided in this application embodiment;

[0053] Figure 2 A schematic diagram of the registration device for whole-slice pathological images provided in an embodiment of this application;

[0054] Figure 3 This is a schematic diagram of the registration device for whole-slice pathological images provided in an embodiment of this application.

[0055] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.

[0058] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0059] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0060] Figure 1 This is a schematic flowchart illustrating a registration method for whole-section pathological images provided in an embodiment of this application. Figure 1 As shown, the registration method for whole-slice pathological images provided in this embodiment includes:

[0061] S1: Obtain whole-section pathological images and preprocess the obtained whole-section pathological images.

[0062] Among them, whole-section pathological images can be, for example, high-resolution pathological sections of breast cancer tissue.

[0063] Specifically, the acquired whole-slice pathological images are preprocessed, including: converting the acquired whole-slice pathological images into grayscale images; performing contrast enhancement processing on the grayscale images to generate enhanced grayscale images; and performing multi-scale downsampling on the enhanced grayscale images to generate an image pyramid structure with multiple resolution levels.

[0064] It is understandable that different staining methods may be used to obtain the full-slice image, and converting the color full-slice image into a grayscale image can help simplify subsequent calculations.

[0065] Furthermore, to improve the robustness and accuracy of subsequent feature point detection, a Contrast-Limited Adaptive Histogram Equalization (CLAHE) method is employed to enhance the grayscale image. CLAHE performs histogram equalization on local regions of the image separately and limits the degree of contrast amplification, effectively enhancing image details while avoiding excessive noise amplification. This helps to highlight fine structures in cases of complex tissue structures or low original contrast, thereby improving the detectability of feature points.

[0066] Furthermore, since the original resolution of the full-slice image is extremely high, direct processing is computationally expensive and inefficient. Therefore, the enhanced grayscale image is downsampled to generate a multi-scale image pyramid structure. This means the image is scaled down to multiple different resolution levels (e.g., from Level 0 to Level 7). Each level represents the image representation at a different scale. This method first estimates the global alignment at a coarse (low-resolution) level, and then progressively optimizes the precise local alignment at finer (high-resolution) levels, thereby improving computational efficiency and helping to avoid getting trapped in local optima.

[0067] S2: Construct a global affine transformation matrix, and perform affine transformation on the preprocessed whole-slice pathological image using the global affine transformation matrix to obtain a preliminary registration image.

[0068] Specifically, the global affine transformation matrix is ​​constructed, including:

[0069] S21: A corner detection algorithm is used to extract multiple feature points from the preprocessed whole-slice pathological image.

[0070] Specifically, for each candidate pixel in the preprocessed whole-slice pathological image, a circular neighborhood centered on the candidate pixel is defined; the gray values ​​of the candidate pixel and other pixels in the circular neighborhood are compared to determine whether the gray value difference between the two points is greater than a preset gray value difference threshold; if the number of pixels in the circular neighborhood whose gray value difference is greater than the preset gray value difference threshold exceeds the preset threshold, then the candidate pixel is used as a feature point.

[0071] For example, the FAST algorithm is used for feature point detection. FAST is a highly efficient corner detector that quickly identifies corners by comparing the gray values ​​of pixels in a circular neighborhood around a given pixel. The detected feature points typically correspond to locations in the image with significant local gray-level changes, such as edge intersections or high curvature points. These feature points constitute key structures in the image and serve as alignment anchors in subsequent descriptor calculations and feature matching.

[0072] S22: Determine the descriptor for each feature point using a preset frequency domain filter bank.

[0073] Understandably, in order for corresponding feature points in different images to be correctly matched, a descriptor needs to be calculated for each extracted feature point. This descriptor should be able to capture the characteristics of its local neighborhood and be robust to common image changes (such as illumination and rotation).

[0074] In an optional embodiment, a Log-Gabor filter bank is used to construct local descriptors for feature points. The descriptor construction process aims to encode the complex image information surrounding the feature points into a compact and discriminative vector.

[0075] Specifically, for a size of (y img ×x img For a grayscale image, first calculate its Log-Gabor filter response at s different scales and o different directions. The purpose is to capture local structural information at different frequencies and directions. At each pixel (x,y), calculate the sum of the magnitudes of its Log-Gabor filter response at all s scales in the j-th direction, denoted as CS(x,y,j):

[0076]

[0077] Among them, EO i,j (x,y) is the complex response value of the Log-Gabor filter at (x,y) at the i-th scale and j-th direction. It is calculated by phase consistency and helps to enhance the robustness to illumination changes. CS(x,y,j) reflects the overall energy response of point (x,y) in direction j.

[0078] Understandably, by finding the direction index MIM(x,y) that maximizes the value of CS(x,y,j), the local dominant direction of each pixel (x,y) can be determined, as shown below:

[0079]

[0080] S2201: For each feature point in the preprocessed whole-slice pathological image, define a square neighborhood centered on the feature point.

[0081] Understandably, for each extracted key feature point, focusing on its square neighborhood of radius R is beneficial for subsequent systematic analysis of the information within this neighborhood.

[0082] S2202: Use the meshgrid function to generate a relative grid coordinate matrix within a square neighborhood.

[0083] Understandably, the generated relative grid coordinate matrix contains the position information of each relative coordinate point within the square neighborhood.

[0084] S2203: Determine the logarithmic amplitude of the feature point based on the Euclidean distance from each relative coordinate point in the square neighborhood to the center point.

[0085] Specifically, calculate the Euclidean distance from each relative coordinate point in the neighborhood to the center point (0,0), and take its base-2 logarithm, satisfying the following formula:

[0086]

[0087] Among them, Log amp For logarithmic amplitude, (x) offset ,y offset ) represents a relative coordinate point, XX represents the X coordinate of each relative coordinate point in the neighborhood, and YY represents the Y coordinate of each relative coordinate point in the neighborhood.

[0088] S2204: Determine the angle of the feature point based on the angle of each relative coordinate point relative to the center point.

[0089] Specifically, the angle of each relative coordinate point relative to the center point is calculated to capture direction information, satisfying the following formula:

[0090]

[0091] Here, mod is the modulo operator.

[0092] S2205: Discretize the logarithmic amplitude of the feature points according to the preset radius threshold to obtain the discrete amplitude index.

[0093] For example, the logarithmic amplitude Logamp The value is discretized based on preset radius thresholds r1, r2, r3, r4 to obtain the discrete amplitude index A. bin This is equivalent to dividing the neighborhood of a feature point into 5 concentric ring regions, each region corresponding to a feature A. bin Index values ​​(1 to 5) are represented as follows:

[0094]

[0095] S2206: Discretize the angle of the relative coordinates to obtain discrete direction indices.

[0096] Specifically, the continuous angle θ is quantized to N. bin In a discrete orientation box, the orientation index θ is obtained. bin This is equivalent to dividing the neighborhood into multiple sectors, satisfying the following formula:

[0097]

[0098] The round function performs a rounding operation on a numerical value.

[0099] S2207: Construct a multidimensional histogram based on the discrete amplitude index and discrete direction index, and use the constructed multidimensional histogram as the descriptor of the feature point.

[0100] Understandably, descriptors are designed to statistically analyze the distribution of points within a neighborhood across different rings and directions.

[0101] First, for each key point KPS(k) = (x k ,y k ), where k = 1,...,M (M is the number of keypoints), extract a keypoint from the image with (x k ,y k A local patch centered at () with radius R is represented as follows:

[0102] P={(x,y)∣(xx k ) 2 +(yy k ) 2 ≤R 2}

[0103] The central ring histogram C(g) is used to count the number of neighborhood points falling into the central ring, and is specifically represented as follows:

[0104]

[0105] Where g∈MIM P δ(·) is an indicator function; it is 1 if the condition is true, and 0 otherwise.

[0106] Outer ring histogram H(r) i ,θ bin_val ), used to count the number of neighbors that simultaneously belong to the r-th node. i a ring r i ∈{2,...,5} and its quantization direction falls into the θ-th position. bin_val Each direction box θ bin_val ∈{1,...,N bin The number of points in the array captures both radial and angular structural information, as shown below:

[0107]

[0108] The final descriptor DES is formed by concatenating the two histograms mentioned above (the vectorized forms of C and H, Vec(H)), as shown below:

[0109] DES = concatenate(C, Vec(H))

[0110] S2208: Gaussian weight the discrete amplitude indices of each point in the square neighborhood, and construct a weighted direction histogram about the discrete direction indices based on the Gaussian weighted discrete amplitude indices.

[0111] Understandably, to make the descriptor insensitive to image rotation, it needs to be rotated normalized. This is achieved by determining a canonical direction (main direction) for the neighborhood of feature points and adjusting the computation or representation of the descriptor relative to this main direction.

[0112] First, for each point A in the neighborhood... bin Gaussian weighted values ​​W(x) offset ,y offset This is to highlight the contribution of the central region to the calculation of the main direction.

[0113]

[0114] W(x,y)=A bin (x,y)·Gauss(x,y)

[0115] Then, construct the quantization angle θ bin The weighted directional histogram h(j):

[0116]

[0117] The direction box j here corresponds to θ bin The value of .

[0118] S2209: Smooth the weighted orientation histogram, and perform peak detection and interpolation on the smoothed weighted orientation histogram to determine the main orientation of the feature point neighborhood.

[0119] Understandably, smoothing the histogram h(j) to obtain h′(j) helps improve the stability of the principal direction estimation. The principal direction DM can be obtained with a more accurate angle by peak detection and interpolation of h′(j), specifically satisfying the following formula:

[0120]

[0121] Where, θ peak_idx This is the box index corresponding to the peak value of h′(j). The subscript `wrap` here refers to the "wrap-around" index. This is useful when dealing with periodic data θ, such as angles. peak_idx In some cases, the boundaries of the histogram require special handling. For example, if a directional histogram has N... bin Each bin has an index from 0 to N. bin -1, then the 0th box and the Nth box bin -1 boxes are adjacent. h(j-1) wrap When j=0, the last box (i.e., N) will be visited. bin -1), while h(j+1) wrap The first bin (i.e., h(0)) is visited when j is the index of the last bin. This wraparound processing ensures the correctness of smoothing or neighborhood calculations at histogram edges.

[0122] S2210: Based on the determined main direction, the descriptor of the feature points is cyclically shifted to obtain the rotated and normalized descriptor.

[0123] Understandably, based on the determined principal direction DM, the descriptor DES calculated in the above steps is cyclically shifted. This means adjusting the "starting" direction of the histogram to the principal direction DM, so that even if the image is rotated, the calculated descriptor vector will remain consistent as long as the feature points and their neighborhood structures are the same. For example, if DM corresponds to the u-th orientation box, then the orientation axis in H will be cyclically shifted by u units.

[0124] S23: Match the descriptors in the target image and the preprocessed whole-slice pathological image according to similarity to obtain multiple candidate matching pairs.

[0125] The target image is the image used as a registration reference.

[0126] Specifically, for each feature point in the preprocessed whole-slice pathological image, the feature point in the target image that has the most similar descriptor (e.g., the smallest Euclidean distance) is searched as a candidate match.

[0127] S24: The fast sampling consensus algorithm is used to calculate the transformation parameters based on multiple candidate matching pairs, and the global affine transformation matrix is ​​determined based on the obtained transformation parameters.

[0128] Understandably, the Fast Sample Consensus (FSC) algorithm is used to calculate the transformation parameters based on multiple candidate matching pairs, and the global affine transformation matrix M is determined based on the obtained transformation parameters. affine .

[0129] This is understandable; the resulting global affine transformation matrix M... affine This describes a global linear transformation (including rotation, scaling, translation, and shearing) from the source image coordinate system to the target image coordinate system. This matrix will be used to perform preliminary global alignment of the source image, making it aligned with the target image overall. The source image represents the whole-slice pathological image after preprocessing in step S1.

[0130] S3: Construct a displacement field based on the global affine transformation matrix, optimize the constructed displacement field, and perform displacement transformation on the preliminary registered image according to the optimized displacement field to obtain the final registered image.

[0131] Specifically, using the global affine transformation matrix M affine An initial displacement field DF is constructed for the non-rigid registration stage. For any pixel p in the source image... s =(x s ,y s ) T The corresponding points after affine transformation satisfy

[0132] Initial displacement field D initial At each pixel (x) s ,y s The displacement vector of ) is defined as:

[0133]

[0134] Where, p′ s (1:2) indicates taking p′ s The first two coordinate components. Initial displacement field D initial It contains global alignment information, and subsequent optimizations will be based on this information and will involve local adjustments.

[0135] Furthermore, to improve optimization efficiency and robustness, non-rigid registration is performed within a multi-scale pyramid framework, from coarse to fine. Specifically, for the source image I... S and target image I TConstruct image pyramids separately (e.g., from the coarsest level L) max To the finest level L0), the initial displacement field D initial It is also adjusted accordingly to the coarsest level. Then, from the coarsest level L... max Initially, the displacement field is optimized layer by layer towards a finer level. At each level L, the optimization objective is to find a residual displacement field ΔD. L This allows the displacement field obtained by passing it from and upsampling it from the previous level to... Compared with the current level optimization ΔD L The sum (i.e., the total displacement field) It can best align the source image of the current layer. and target image The optimization objective function typically includes a similarity metric E. sim And a regularization term R(ΔD) L ):

[0136]

[0137] Where, λ L This is a weighting factor at level L, used to balance the similarity term E. sim and regularization term R(ΔD) L The importance of this relationship lies in controlling the trade-off between image matching accuracy and deformation smoothness.

[0138] Optimized This is used to update the total displacement field, which is then upsampled to the next finer level L-1 as the initial displacement for optimization at that level. This process is repeated iteratively until the finest level is reached.

[0139] Understandably, in the displacement field optimization process, a regularization term R(D) is introduced to ensure its smoothness and physical rationality, and to avoid unnatural excessive distortion. In this embodiment, relative diffusion regularization is adopted, which can adaptively adjust the strength of the smoothing constraint according to the local structural characteristics of the image: allowing a larger displacement gradient at the edges of the image structure, while forcing a stronger smoothing in flat regions, specifically satisfying the following formula:

[0140]

[0141] in,‖·‖ F It is the Frobenius norm. It is the Jacobian matrix of the displacement field. The weights w(x) depend on the image content, for example... in α is the gradient magnitude of the target image at point x, and α is a positive scaling parameter that controls the sensitivity of the weight w(x) to the gradient magnitude. k is an exponent, usually 1 or 2, which adjusts how the gradient magnitude affects the weights.

[0142] Understandably, the core of non-rigid registration lies in minimizing a total loss function L(D) that includes a similarity measure and a regularization term. This embodiment uses Local Normalized Cross-Correlation (Local-NCC) as the similarity measure E. sim For each point x in the image, a small window W is placed around it. x Internal calculation of the NCC value between the deformed source image block and the target image block:

[0143]

[0144] in, This refers to the source image after deformation by the current displacement field D. I S It is the source image, I T The target image is x, where x is the number of pixels and W is the number of pixels. x p is a small neighborhood window centered at pixel x, where p is the window W. x Any pixel within, This indicates that the source image after deformation by the current displacement field D is displayed in window W. x Average gray level within, This indicates that the target image is in window W. x The average gray level within; the total loss function is defined as:

[0145]

[0146] Here, L(D) is the total loss value with respect to the displacement field D. The optimization objective is to find the displacement field D that minimizes L(D); D(x) is the displacement vector at pixel x.

[0147] The ADAM (Adaptive Moment Estimation) optimizer is used to iteratively update the parameters of the displacement field D until the loss function converges.

[0148] Furthermore, after multi-scale iterative optimization, the final dense displacement field D is obtained. final Using this displacement field, the original high-resolution source image I... S The image is deformed to obtain the final registered image.

[0149] this Image and target image I T Achieving a high degree of alignment in both structure and details.

[0150] The registration method for whole-slice pathological images provided in this application involves acquiring a whole-slice pathological image and preprocessing it; constructing a global affine transformation matrix and performing an affine transformation on the preprocessed whole-slice pathological image using the global affine transformation matrix to obtain a preliminary registration image; constructing a displacement field based on the global affine transformation matrix and optimizing the constructed displacement field; and performing a displacement transformation on the preliminary registration image based on the optimized displacement field to obtain a final registration image, thereby improving the accuracy of the registration result.

[0151] Figure 2 This is a schematic diagram of the registration device for whole-slice pathological images provided in an embodiment of this application. Figure 2 As shown, the registration device 200 for whole-slice pathological images provided in this embodiment includes:

[0152] Module 201 is used to acquire whole-slice pathological images;

[0153] The first processing module 202 is used to preprocess the acquired whole-slice pathological images;

[0154] The second processing module 203 is used to construct a global affine transformation matrix, and to perform affine transformation on the preprocessed whole-slice pathological image through the global affine transformation matrix to obtain a preliminary registration image.

[0155] The second processing module 203 is also used to construct a displacement field based on the global affine transformation matrix, optimize the constructed displacement field, and perform displacement transformation on the preliminary registered image according to the optimized displacement field to obtain the final registered image.

[0156] The registration device for whole-slice pathological images provided in this embodiment can perform the registration method for whole-slice pathological images provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0157] Figure 3 This is a schematic diagram of the registration device for whole-slice pathological images provided in an embodiment of this application. Figure 3 As shown in the embodiment of this application, the registration device for whole-slice pathological images 300 includes: a receiver 301, a transmitter 302, a processor 303, and a memory 304.

[0158] Receiver 301 is used to receive instructions and data;

[0159] Transmitter 302 is used to send commands and data;

[0160] Memory 304 is used to store computer-executed instructions;

[0161] Processor 303 is used to execute computer execution instructions stored in memory 304 to implement the various steps of the whole-slice pathological image registration method in the above embodiments. For details, please refer to the relevant descriptions in the foregoing embodiments of the whole-slice pathological image registration method.

[0162] Optionally, the memory 304 can be either standalone or integrated with the processor 303.

[0163] When the memory 304 is set up independently, the electronic device also includes a bus for connecting the memory 304 and the processor 303.

[0164] This application embodiment also provides a computer storage medium storing computer execution instructions. When the processor executes the computer execution instructions, it implements the registration method of the whole-slice pathological image as described above by the registration device for whole-slice pathological images.

[0165] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0166] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0167] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A registration method for whole-section pathological images, characterized in that, The method includes: Obtain whole-section pathological images and preprocess the obtained whole-section pathological images; A global affine transformation matrix is ​​constructed, and the preprocessed whole-slice pathological image is subjected to affine transformation using the global affine transformation matrix to obtain a preliminary registration image; The construction of the global affine transformation matrix includes: For each candidate pixel in the preprocessed whole-slice pathological image, a circular neighborhood centered on the candidate pixel is defined. The gray values ​​of candidate pixels within a circular neighborhood are compared with those of other pixels to determine whether the gray value difference between the two points is greater than a preset gray value difference threshold. If the number of pixels in the circular neighborhood whose grayscale difference is greater than the preset grayscale difference threshold exceeds the preset threshold, then the candidate pixels are used as feature points. For each feature point in the preprocessed whole-slice pathological image, a square neighborhood centered on the feature point is defined; Use the meshgrid function to generate a relative grid coordinate matrix within a square neighborhood; The logarithmic amplitude of the feature point is determined based on the Euclidean distance from each relative coordinate point in the square neighborhood to the center point. The angle of the feature point is determined based on the angle of each relative coordinate point relative to the center point; The logarithmic amplitude of the feature points is discretized according to a preset radius threshold to obtain a discrete amplitude index; Discretize the angles of the feature points to obtain discrete direction indices; A multidimensional histogram is constructed based on the discrete amplitude index and the discrete direction index, and the constructed multidimensional histogram is used as the descriptor of the feature point. Gaussian weighting is applied to the discrete amplitude indices of each point within the square neighborhood, and a weighted direction histogram about the discrete direction indices is constructed based on the Gaussian weighted discrete amplitude indices. The weighted orientation histogram is smoothed, and peak detection and interpolation are performed on the smoothed weighted orientation histogram to determine the main orientation of the feature point neighborhood; Based on the determined main direction, the descriptor of the feature points is cyclically shifted to obtain the rotation-normalized descriptor. Descriptors in the target image and the preprocessed whole-slice pathological image are matched based on similarity to obtain multiple candidate matching pairs; The fast sampling consensus algorithm is used to calculate the transformation parameters based on multiple candidate matching pairs, and the global affine transformation matrix is ​​determined based on the obtained transformation parameters. A displacement field is constructed based on a global affine transformation matrix, and the constructed displacement field is optimized. The displacement transformation is then applied to the preliminary registered image based on the optimized displacement field to obtain the final registered image.

2. The method according to claim 1, characterized in that, The process of constructing a displacement field based on a global affine transformation matrix, optimizing the constructed displacement field, and performing a displacement transformation on the preliminary registered image based on the optimized displacement field to obtain the final registered image includes: The initial displacement field is constructed using an affine transformation matrix; The initial displacement field is optimized step by step according to the preset optimization objective function to obtain the optimized displacement field. The preset optimization objective function includes: a similarity metric term and a regularization term. The optimized displacement field is used to perform displacement transformation on the deformation of the initial registered image to obtain the final registered image.

3. The method according to claim 1, characterized in that, The preprocessing of the acquired whole-section pathological images includes: Convert the acquired whole-section pathological images into grayscale images; Perform contrast enhancement processing on a grayscale image to generate an enhanced grayscale image; The enhanced grayscale image is downsampled at multiple scales to generate an image pyramid structure with multiple resolution levels.

4. A registration device for whole-section pathological images, characterized in that, The device includes: The acquisition module is used to acquire whole-slice pathological images; The first processing module is used to preprocess the acquired whole-slice pathological images; The second processing module is used to construct a global affine transformation matrix, and to perform affine transformation on the preprocessed whole-slice pathological image through the global affine transformation matrix to obtain a preliminary registration image. The construction of the global affine transformation matrix includes: For each candidate pixel in the preprocessed whole-slice pathological image, a circular neighborhood centered on the candidate pixel is defined. The gray values ​​of candidate pixels within a circular neighborhood are compared with those of other pixels to determine whether the gray value difference between the two points is greater than a preset gray value difference threshold. If the number of pixels in the circular neighborhood whose grayscale difference is greater than the preset grayscale difference threshold exceeds the preset threshold, then the candidate pixels are used as feature points. For each feature point in the preprocessed whole-slice pathological image, a square neighborhood centered on the feature point is defined; Use the meshgrid function to generate a relative grid coordinate matrix within a square neighborhood; The logarithmic amplitude of the feature point is determined based on the Euclidean distance from each relative coordinate point in the square neighborhood to the center point. The angle of the feature point is determined based on the angle of each relative coordinate point relative to the center point; The logarithmic amplitude of the feature points is discretized according to a preset radius threshold to obtain a discrete amplitude index; Discretize the angles of the feature points to obtain discrete direction indices; A multidimensional histogram is constructed based on the discrete amplitude index and the discrete direction index, and the constructed multidimensional histogram is used as the descriptor of the feature point. Gaussian weighting is applied to the discrete amplitude indices of each point within the square neighborhood, and a weighted direction histogram about the discrete direction indices is constructed based on the Gaussian weighted discrete amplitude indices. The weighted orientation histogram is smoothed, and peak detection and interpolation are performed on the smoothed weighted orientation histogram to determine the main orientation of the feature point neighborhood; Based on the determined main direction, the descriptor of the feature points is cyclically shifted to obtain the rotation-normalized descriptor. Descriptors in the target image and the preprocessed whole-slice pathological image are matched based on similarity to obtain multiple candidate matching pairs; The fast sampling consensus algorithm is used to calculate the transformation parameters based on multiple candidate matching pairs, and the global affine transformation matrix is ​​determined based on the obtained transformation parameters. The second processing module is also used to construct a displacement field based on the global affine transformation matrix, optimize the constructed displacement field, and perform displacement transformation on the preliminary registered image according to the optimized displacement field to obtain the final registered image.

5. A registration device for whole-section pathological images, characterized in that, The device includes: Memory; processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the registration method for whole-slice pathological images as described in any one of claims 1-3.

6. A computer storage medium, characterized in that, The computer storage medium stores computer execution instructions, which, when executed by a processor, are used to implement the registration method for whole-slice pathological images as described in any one of claims 1-3.

Images