Debugging method and system for factory parameters of digital pathological section scanner and medium
By performing coarse registration, fine registration, and thermal analysis on WSI images from digital pathology slide scanners, and automatically adjusting factory parameters, the problem of low efficiency in existing technologies is solved, and efficient parameter adjustment is achieved.
Patent Information
- Application Number
- CN202510872032.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-26
- Publication Date
- 2025-10-31
AI Technical Summary
In existing technologies, the factory parameter adjustment of digital pathology slide scanners is inefficient, and manual inspection of image quality is time-consuming and labor-intensive, making it difficult to meet the needs of large-scale production.
By acquiring the WSI image to be inspected and the reference WSI image for coarse and fine registration, and combining foreground segmentation and thermal analysis, the factory parameters are automatically adjusted.
It improves the efficiency of setting up factory parameters for digital pathology slide scanners, reduces the cost of manual intervention, and increases production efficiency.
Smart Images
Figure CN120876370A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a method, system, and medium for adjusting the factory parameters of a digital pathology slide scanner. Background Technology
[0002] The core equipment in the modern pathology industry is the digital pathology slide scanner, and a high-performance device often needs to excel in image quality. The core component of a digital pathology slide scanner is a high-magnification microscope objective equipped with a digital camera, which first focuses the image using algorithms. Then, it uses mechanical structures to control movement, traversing every field of view of the specimen slide and taking pictures. Finally, all images are stitched together to form a complete WSI (Whole Slide Image) file.
[0003] Even with identical manufacturing processes, accumulated differences are difficult to avoid between different devices; imaging results will inevitably differ even with identical parameters but different equipment. Typically, after equipment production, engineers manually inspect the imaging quality, comparing digital images with microscopic images, focusing on issues such as blurriness and abrupt stitching boundaries, before deciding how to adjust parameters. This process usually requires repeated adjustments to achieve the desired results, demanding significant time and effort from engineers. As demand increases and production scales up, the manpower and time costs of manual adjustments become unacceptable. Therefore, improving the efficiency of adjusting the factory parameters of digital pathology slide scanners has become a pressing issue.
[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention The main objective of this invention is to provide a method, system, and medium for adjusting the factory parameters of a digital pathology slide scanner, aiming to address the technical problem of improving the efficiency of adjusting the factory parameters of a digital pathology slide scanner.
[0005] To achieve the above objectives, the present invention provides a method for adjusting the factory parameters of a digital pathology slide scanner, the method comprising: The WSI image of the pathological slide to be examined is acquired by a digital pathological slide scanner to be verified, and a reference WSI image of the pathological slide is obtained by a standard digital pathological slide scanner. The reference WSI image and the WSI image to be inspected are coarsely registered to obtain a coarsely registered WSI image. Foreground segmentation is performed on the reference WSI image and the coarse registration WSI image respectively to obtain a reference foreground region image and a coarse registration foreground region image; The reference foreground region image and the coarsely registered foreground region image are finely registered to obtain a finely registered WSI image; Thermal analysis was performed on the precisely registered WSI image, and the factory parameters of the digital pathology slide scanner to be verified were adjusted based on the thermal analysis results.
[0006] Optionally, the step of coarsely registering the reference WSI image and the WSI image to be inspected to obtain a coarsely registered WSI image includes: SIFT key points corresponding to the reference WSI image and the WSI image to be inspected are extracted using a SIFT detector. Select coarse matching feature point pairs from multiple SIFT key points in the reference WSI image and multiple SIFT key points in the WSI image to be inspected; Calculate the coarse registration transformation matrix based on the coarse matching feature point pairs; Based on the coarse registration transformation matrix, the WSI image to be detected is mapped to the coordinates of the reference WSI image to obtain the coarse registration WSI image.
[0007] Optionally, the step of selecting coarse matching feature point pairs from multiple SIFT keypoints in the reference WSI image and multiple SIFT keypoints in the WSI image to be inspected includes: The SIFT detector extracts the descriptors corresponding to each SIFT keypoint in the reference WSI image and the descriptors corresponding to each SIFT keypoint in the WSI image to be tested. Calculate the distance between the descriptors of each SIFT keypoint in the reference WSI image and the descriptors of each SIFT keypoint in the WSI image to be inspected; Based on the distance, coarse matching feature point pairs are selected from multiple SIFT keypoints in the reference WSI image and multiple SIFT keypoints in the WSI image to be inspected.
[0008] Optionally, the step of performing foreground segmentation on the reference WSI image and the coarsely registered WSI image respectively to obtain a reference foreground region image and a coarsely registered foreground region image includes: Multiple WSI image samples were obtained, and contour annotations were performed on each WSI image sample to obtain multiple WSI image annotation samples and the corresponding masks for each WSI image annotation sample. A tissue segmentation training set is constructed based on each WSI image annotation sample and the corresponding mask for each WSI image annotation sample. An initial image segmentation model is trained using the tissue segmentation training set, and the initial image segmentation model is built based on the deeplabv3+ framework; The reference WSI image and the coarsely registered WSI image are respectively input into the trained image segmentation model for foreground segmentation, and the foreground mask of the reference WSI image and the foreground mask of the coarsely registered WSI image are output. A reference foreground region image is obtained based on the reference WSI image and the foreground mask of the reference WSI image, and a coarse registration foreground region image is obtained based on the coarse registration WSI image and the foreground mask of the coarse registration WSI image.
[0009] Optionally, the step of performing fine registration between the reference foreground region image and the coarsely registered foreground region image to obtain a finely registered WSI image includes: The reference foreground region image and the coarsely registered foreground region image are respectively divided into image blocks; Feature point matching is performed based on the segmented reference foreground region image blocks and the coarsely registered foreground region image blocks. Calculate the distance between the descriptors of multiple SIFT keypoints within the reference foreground region image block and the descriptors of multiple SIFT keypoints within each coarsely registered foreground region image block; Based on the distance, fine-matching feature point pairs are selected from multiple SIFT keypoints within each reference foreground region image block and multiple SIFT keypoints within each coarse-registered foreground region image block. Calculate the fine registration transformation matrix based on the fine-matched feature point pairs; Based on the fine registration transformation matrix, the coarse registration foreground region image is mapped to the coordinates of the reference foreground region image to obtain the fine registration WSI image.
[0010] Optionally, calculating the fine registration transformation matrix based on the fine-matched feature point pairs includes: Construct a linear equation for the matching point pair based on the coordinates and homography matrix of the finely matched feature point pair; The fine registration transformation matrix is obtained by solving the linear equation of the matching point pair using the least squares method.
[0011] Optionally, the thermal analysis of the finely registered WSI image includes: Extract the three-channel reference grayscale image and the three-channel finely registered grayscale image corresponding to the reference foreground region image and the finely registered WSI image respectively, and calculate the three-channel grayscale difference based on the three-channel reference grayscale image and the three-channel finely registered grayscale image; Calculate the gradients in different directions corresponding to the reference foreground region image and the finely registered WSI image respectively, and calculate the reference intensity matrix and the finely registered intensity matrix corresponding to the reference foreground region image and the finely registered WSI image respectively based on the gradients in different directions; The intensity difference between each pixel in the reference foreground region image and each pixel in the finely registered WSI image is calculated based on the reference intensity matrix and the finely registered intensity matrix. The pixels corresponding to the three-channel grayscale difference and the pixels corresponding to the intensity difference are mapped to a preset color space to obtain a three-channel grayscale difference heatmap and a pixel intensity difference heatmap. The three-channel grayscale difference heatmap and the pixel intensity difference heatmap are superimposed on the WSI image to be inspected for thermal analysis.
[0012] Optionally, after performing thermal analysis on the finely registered WSI image, the method further includes: Quantitative analysis is performed on the reference foreground region image and the finely registered WSI image; The factory parameters of the digital pathology slide scanner to be verified were adjusted based on the results of thermal analysis and the quantitative analysis.
[0013] Furthermore, to achieve the above objectives, the present invention also proposes a system for adjusting the factory parameters of a digital pathology slide scanner, the system comprising: The acquisition module is used to acquire the WSI image of the pathological slide to be examined through the digital pathological slide scanner to be verified, and to obtain the reference WSI image of the pathological slide. The registration module is used to coarsely register the reference WSI image and the WSI image to be inspected to obtain a coarsely registered WSI image. The segmentation module is used to perform foreground segmentation on the reference WSI image and the coarse registration WSI image respectively, to obtain a reference foreground region image and a coarse registration foreground region image; The registration module is further configured to perform fine registration of the reference foreground region image and the coarsely registered foreground region image to obtain a finely registered WSI image; The debugging module is used to perform thermal analysis on the finely registered WSI image and to debug the factory parameters of the digital pathology slide scanner to be verified based on the thermal analysis results.
[0014] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a debugging program for the factory parameters of a digital pathology slide scanner. When the debugging program for the factory parameters of the digital pathology slide scanner is executed by a processor, it implements the steps of the debugging method for the factory parameters of the digital pathology slide scanner as described above.
[0015] This invention first acquires a WSI image of a pathological slide to be tested using a digital pathology slide scanner to be calibrated, and obtains a reference WSI image of the pathological slide. Then, the reference WSI image and the WSI image to be tested are coarsely registered to obtain a coarsely registered WSI image. Foreground segmentation is then performed on both the reference WSI image and the coarsely registered WSI image to obtain a reference foreground region image and a coarsely registered foreground region image, respectively. Next, the reference foreground region image and the coarsely registered foreground region image are finely registered to obtain a finely registered WSI image. Finally, thermal analysis is performed on the finely registered WSI image, and the factory parameters of the digital pathology slide scanner to be calibrated are adjusted based on the thermal analysis results. Compared to existing technologies where engineers need to manually check the imaging quality of the equipment and adjust parameters by comparing the differences between digital images and microscopic images, resulting in low efficiency in adjusting the scanner's factory parameters, this invention performs coarse and fine registration of the WSI image to be tested with the reference WSI image, and then performs thermal analysis on the finely registered WSI image. This provides an intuitive thermal map to assist engineers in adjusting the equipment, improving the efficiency of adjusting the factory parameters of the digital pathology slide scanner. Attached Figure Description
[0016] Figure 1 This is a schematic diagram of the structure of the debugging device for the factory parameters of the digital pathology slide scanner in the hardware operating environment involved in the embodiments of the present invention; Figure 2 This is a flowchart illustrating the first embodiment of the debugging method for the factory parameters of the digital pathology slide scanner of the present invention. Figure 3 This is a structural block diagram of the first embodiment of the debugging system for the factory parameters of the digital pathology slide scanner of the present invention.
[0017] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0018] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.
[0019] Reference Figure 1 , Figure 1 This is a schematic diagram of the debugging equipment structure for the factory parameters of the digital pathology slide scanner in the hardware operating environment involved in the embodiments of the present invention.
[0020] like Figure 1As shown, the device for debugging the factory parameters of the digital pathology slide scanner may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk storage device. Optionally, the memory 1005 may also be a storage system independent of the aforementioned processor 1001.
[0021] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the factory settings of the digital pathology slide scanner. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0022] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a debugging program for the factory parameters of the digital pathology slide scanner.
[0023] exist Figure 1 In the digital pathology slide scanner factory parameter debugging device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the digital pathology slide scanner factory parameter debugging device of the present invention can be set in the digital pathology slide scanner factory parameter debugging device. The digital pathology slide scanner factory parameter debugging device calls the digital pathology slide scanner factory parameter debugging program stored in the memory 1005 through the processor 1001 and executes the digital pathology slide scanner factory parameter debugging method provided in the embodiment of the present invention.
[0024] This invention provides a method for adjusting the factory parameters of a digital pathology slide scanner, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the debugging method for the factory parameters of the digital pathology slide scanner of the present invention.
[0025] In this embodiment, the method for adjusting the factory parameters of the digital pathology slide scanner includes the following steps: Step S10: Acquire the WSI image of the pathological slide to be examined using a digital pathological slide scanner to be verified, and obtain the reference WSI image of the pathological slide using a standard digital pathological slide scanner.
[0026] It is easy to understand that the execution subject of this embodiment can be a debugging system for the factory parameters of a digital pathology slide scanner with functions such as data processing, network communication and program operation, or other computer devices with similar functions. This embodiment does not limit it.
[0027] It should be noted that the reference WSI image (i.e., W_r) is a WSI image acquired by a standard digital pathology slide scanner (i.e., a calibrated digital pathology slide scanner). The acquired WSI image is the standard WSI image corresponding to the WSI image to be examined. The WSI image to be examined (i.e., W_t) and the reference WSI image (i.e., W_r) can be WSI images at a preset magnification, such as WSI images at 20x.
[0028] Step S20: Perform coarse registration between the reference WSI image and the WSI image to be inspected to obtain a coarsely registered WSI image.
[0029] Furthermore, the reference WSI image and the WSI image to be inspected are coarsely registered to obtain the coarsely registered WSI image. The processing method is as follows: SIFT key points corresponding to the reference WSI image and the WSI image to be inspected are extracted by a Scale-Invariant Feature Transform (SIFT) detector; coarse matching feature point pairs are selected from multiple SIFT key points in the reference WSI image and multiple SIFT key points in the WSI image to be inspected; coarse registration transformation matrix is calculated based on coarse matching feature point pairs; and the WSI image to be inspected is mapped to the coordinates of the reference WSI image based on the coarse registration transformation matrix to obtain the coarsely registered WSI image.
[0030] The process of selecting coarse matching feature point pairs from multiple SIFT keypoints in the reference WSI image and multiple SIFT keypoints in the WSI image to be tested is as follows: The descriptors corresponding to each SIFT keypoint in the reference WSI image and the descriptors corresponding to each SIFT keypoint in the WSI image to be tested are extracted using a SIFT detector; the distance between the descriptors of each SIFT keypoint in the reference WSI image and the descriptors of each SIFT keypoint in the WSI image to be tested is calculated; and coarse matching feature point pairs are selected from multiple SIFT keypoints in the reference WSI image and multiple SIFT keypoints in the WSI image to be tested based on the distance.
[0031] In the specific implementation, the SIFT detector is used to extract the SIFT keypoints and descriptors corresponding to W_r and W_t respectively. Then, the distance between the descriptors of each SIFT keypoint in W_r and the descriptors of each SIFT keypoint in W_t is calculated using a brute-force matching method to find matching feature point pairs (i.e., coarse matching feature point pairs). The coarse registration transformation matrix M_rough is calculated using the matching feature point pairs. Finally, W_t is mapped to the coordinates of W_r according to M_r to obtain the coarse registration WSI image (i.e., W_t').
[0032] It should also be noted that the distances between the descriptors of each SIFT keypoint in W_r and the descriptors of each SIFT keypoint in W_t are sorted from near to far. Based on the distance sorting results, a preset percentage of matching feature point pairs are selected from front to back. The preset percentage can be 30%, 40%, etc., and this embodiment does not impose any restrictions.
[0033] The process for calculating the coarse registration transformation matrix based on coarse matching feature point pairs is as follows: construct the linear equation of the matching point pair based on the coordinates and homography matrix of the coarse matching feature point pair; solve the linear equation of the matching point pair using the least squares method to obtain the coarse registration transformation matrix.
[0034] The homography matrix H has the following form:
[0035] in, , , ...... These are the parameters of the matrix, which are used to control the effect of perspective transformation.
[0036] For each pair of coarse-matched feature points ( , )and( , The goal is to use the homography matrix H to identify the feature points of W_t. , ) Transform to feature points of W_r ( , ),Right now:
[0037] After expansion, the following linear equations for matching point pairs are obtained:
[0038] It should also be noted that at least four pairs of coarse-matched feature points are required to solve this problem.
[0039] Represent the above equation using a matrix:
[0040] In the formula, n represents the nth coarse matching feature point.
[0041] It should be understood that the homography matrix H can be calculated using the least squares method, and the calculated homography matrix H is the coarse registration transformation matrix, thus completing the coarse registration of the image.
[0042] Step S30: Perform foreground segmentation on the reference WSI image and the coarse registration WSI image respectively to obtain a reference foreground region image and a coarse registration foreground region image.
[0043] Furthermore, the foreground segmentation of the reference WSI image and the coarsely registered WSI image is performed separately to obtain the reference foreground region image and the coarsely registered foreground region image. The processing method is as follows: multiple WSI image samples are obtained, and tissue contour annotation is performed on each WSI image sample to obtain multiple WSI image annotation samples and the corresponding masks for each WSI image annotation sample; a tissue segmentation training set is constructed based on each WSI image annotation sample and the corresponding mask; an initial image segmentation model is trained using the tissue segmentation training set, and the initial image segmentation model is constructed based on the deeplabv3+ framework; the reference WSI image and the coarsely registered WSI image are input into the trained image segmentation model for foreground segmentation, and the foreground mask (i.e., Mask_r) of the reference WSI image and the foreground mask (i.e., MASK_t') of the coarsely registered WSI image are output; the reference foreground region image is obtained based on the reference WSI image and the foreground mask of the reference WSI image, and the coarsely registered foreground region image is obtained based on the coarsely registered WSI image and the foreground mask of the coarsely registered WSI image.
[0044] In this embodiment, in the specific implementation, the minimum bounding box w_r (i.e., the reference foreground region image of the minimum bounding box) and w_t' (i.e., the coarse registration foreground region image of the minimum bounding box) of the foreground region required for fine registration are obtained by combining W_r, W_t', Mask_r and MASK_t' (the coordinates of w_r and w_t' are consistent with those of W_r and W_t', respectively).
[0045] In the specific implementation, high-resolution pyramid pathological image data (such as WSI in formats like TMAP, SVS, or TIFF, ensuring a rich variety of slide imaging types and covering as many real-world usage scenarios as possible, but without specific requirements for the diagnostic classification of the samples) is collected. The collected high-resolution pyramid pathological image data is then converted into multiple WSI image samples (e.g., 300 WSI image samples), each WSI image sample being a 1x image. Next, an image annotation tool (e.g., Labelme) is used to accurately annotate the tissue contours in the image samples using polygons, obtaining the corresponding mask (the tissue segmentation model only involves background and foreground; all annotated targets belong to the same category). The annotated images (i.e., the WSI image annotation samples) and the mask constitute the tissue segmentation training set D_tis. Based on the obtained dataset D_tis, an initial image segmentation model M_tis (built using the deeplabv3+ framework) is trained. After completion, the model weight file is retained, resulting in the trained image segmentation model.
[0046] Step S40: Perform fine registration on the reference foreground region image and the coarsely registered foreground region image to obtain a finely registered WSI image.
[0047] Furthermore, the process of finely registering the reference foreground region image and the coarsely registered foreground region image to obtain the finely registered WSI image is as follows: the reference foreground region image and the coarsely registered foreground region image are divided into image blocks respectively; feature point matching is performed based on the divided image blocks to obtain finely matched feature point pairs; the fine registration transformation matrix is calculated based on the finely matched feature point pairs; and the coarsely registered foreground region image is mapped to the coordinates of the reference foreground region image based on the fine registration transformation matrix to obtain the finely registered WSI image.
[0048] It should also be noted that the process of obtaining finely matched feature point pairs based on feature point matching of the divided image blocks is as follows: calculate the distance between the descriptors of multiple SIFT keypoints in each reference foreground region image block and the descriptors of multiple SIFT keypoints in the corresponding coarsely registered foreground region image block, wherein the position of the reference foreground region image block is the same as the position of the coarsely registered foreground region image block; select finely matched feature point pairs from the multiple SIFT keypoints in each reference foreground region image block and the multiple SIFT keypoints in each coarsely registered foreground region image block according to the distance.
[0049] In the specific implementation, w_r and w_t' are traversed in (512*512) patches (i.e., image blocks) to obtain the patch set P_r corresponding to w_r and the patch set P_t' corresponding to w_t'. Then, the corresponding SIFT keypoints and descriptors of the patches at the positions in P_r and P_t' are extracted using the SIFT detector. Then, the distance between descriptors is calculated using a brute-force matching method. The distances between all descriptors are sorted from near to far. Based on the distance sorting results, a preset percentage of fine-matching feature point pairs are selected from front to back. The preset percentage can be 30%, 40%, etc.
[0050] In this embodiment, the top 30% of the SIFT keypoint pairs closest to each other from multiple SIFT keypoints in each reference foreground region image block and multiple SIFT keypoints in each coarsely registered foreground region image block are selected as fine-matching feature point pairs for w_r and w_t'. The fine-matching transformation matrix M_fine is calculated using the fine-matching feature point pairs. Finally, w_t' is mapped to w_r according to M_fine to obtain the fine-registered WSI image w_t''.
[0051] It should also be noted that the calculation method for obtaining the fine registration transformation matrix using fine-matched feature point pairs is the same as the calculation method for obtaining the coarse registration transformation matrix using coarse-matched feature point pairs. A linear equation for the matching point pairs is constructed based on the coordinates and homography matrix of the fine-matched feature point pairs; the fine registration transformation matrix is obtained by solving the linear equation for the matching point pairs using the least squares method.
[0052] Step S50: Perform thermal analysis on the finely registered WSI image, and adjust the factory parameters of the digital pathology slide scanner to be verified based on the thermal analysis results.
[0053] Further, three-channel reference grayscale images and three-channel finely registered grayscale images corresponding to the reference foreground region image and the finely registered WSI image are extracted respectively, and the three-channel grayscale difference is calculated based on the three-channel reference grayscale images and the three-channel finely registered grayscale images; gradients in different directions corresponding to the reference foreground region image and the finely registered WSI image are calculated respectively, and reference intensity matrices and finely registered intensity matrices corresponding to the reference foreground region image and the finely registered WSI image are calculated based on the reference intensity matrix and the finely registered intensity matrix; the intensity difference between each pixel in the reference foreground region image and each pixel in the finely registered WSI image is calculated based on the reference intensity matrix and the finely registered intensity matrix; the pixels corresponding to the three-channel grayscale difference and the pixels corresponding to the intensity difference are mapped to a preset color space to obtain a three-channel grayscale difference heatmap and a pixel intensity difference heatmap; the three-channel grayscale difference heatmap and the pixel intensity difference heatmap are superimposed on the WSI image to be inspected to obtain a heatmap, thermal analysis is performed based on the heatmap, and the factory parameters of the digital pathology slide scanner to be verified are adjusted based on the thermal analysis results.
[0054] In the specific implementation, the three-channel grayscale images R_r, G_r, B_r (i.e., the three-channel reference grayscale images) and R_t'', G_t'', B_t'' (i.e., the three-channel finely registered grayscale images) of w_r are extracted respectively. The three-channel grayscale images are subtracted to obtain the three-channel grayscale differences Δr, Δg, and Δb. Using the matplotlib tool, the pixels of Δr, Δg, and Δb are mapped to a preset color space (i.e., the red-blue color space) to obtain the three-channel grayscale difference heatmaps H_r, H_g, and H_b. Positive numbers are mapped to red areas, negative numbers are mapped to blue areas, and the color intensity represents the magnitude of the absolute value of the difference. Finally, H_r, H_g, and H_b are superimposed on W_t with an adjustable fusion ratio to obtain the three-channel fused heatmaps F_r, F_g, and F_b.
[0055] Using 3x3 convolution kernels, the gradients Ix and Iy in the X and Y directions of w_r and w_t'' are calculated respectively. Then, the intensity of each pixel of w_r and w_t'' is calculated using their gradients in each direction to obtain the reference intensity matrix G_r and the fine registration intensity matrix G_t''. The difference between G_r and G_t'' is calculated to obtain the intensity difference ΔG. ΔG is mapped to the red-blue color space to obtain the pixel intensity difference heatmap H_G. Positive values are mapped to red areas (representing that the reference image is clearer than the image to be inspected), and negative values are mapped to blue areas (representing that the reference image is blurrier than the image to be inspected). The color intensity represents the magnitude of the absolute value of the difference. Finally, H_G is superimposed on W_t with an adjustable fusion ratio to obtain the intensity fusion heatmap F_G.
[0056] Horizontal gradient: 3x3 convolution kernel:
[0057] For each pixel (x, y) in images w_r and w_t'', the horizontal gradient Ix is calculated as follows:
[0058] In the formula, (i, j) are the indices of the 3x3 convolution kernel.
[0059] Vertical gradient: Convolution kernel:
[0060] The vertical gradient Iy is calculated as follows:
[0061]
[0062] In the formula, G(x,y) is the intensity of the pixel.
[0063] The thermal map is obtained by combining the three-channel fusion thermal maps F_r, F_g, and F_b with the intensity fusion thermal map F_G. Engineers perform thermal analysis on the thermal map and adjust the factory parameters of the digital pathology slide scanner to be verified based on the results of the thermal analysis.
[0064] In this embodiment, quantitative analysis is performed on the reference foreground region image and the finely registered WSI image; the factory parameters of the digital pathology slide scanner to be verified are adjusted based on the thermal analysis results and the quantitative analysis results.
[0065] The quantitative analysis of the reference foreground region image and the finely registered WSI image is performed by calculating the image quality index UQI and the feature similarity index FSIM between w_r and w_t''.
[0066] UQI calculates image quality by considering the brightness, contrast, and structural information of an image. The advantage of the UQI algorithm is that it is easily computed and applied to various image processing applications. Although it is based on mathematical definitions and does not directly simulate the human visual system, research shows that it has a high degree of consistency with subjective quality assessments.
[0067] UQI calculation formula:
[0068] In the formula, and These are the average brightness values of w_r and w_t'', respectively. and These are the variances of w_r and w_t'', respectively (i.e., the image contrast). It is the covariance between w_r and w_t'' (i.e., the correlation between images).
[0069] The UQI value ranges from [-1, 1]. The closer the value is to 1, the better the image quality; the closer it is to -1, the worse the image quality. When the UQI value is 1, it means that the two images are exactly the same.
[0070] In practical applications, UQI can be used to compare the quality differences between the original image and images that have been compressed, enhanced, or otherwise processed. It is robust to changes in image noise, translation, scaling, rotation, and cropping.
[0071] FSIM is primarily based on two key features: Phase consistency, which describes the symmetry and similarity of structural information in an image; and Gradient magnitude, which reflects changes in image intensity and is used to capture details.
[0072] FSIM calculation steps: 1) Feature Extraction FSIM first extracts gradient information from the image (i.e., w_r and w_t'') using a Laplacian Gaussian (LoG) filter to obtain local features.
[0073] 2) Calculate phase consistency Phase information for each pixel is typically obtained through Fourier transform.
[0074] Based on the phase information, calculate phase consistency (PC):
[0075] Here, Gx and Gy are the gradients of w_r and w_t'', respectively.
[0076] 3) Calculate the gradient magnitude Calculate the gradient magnitude of the image:
[0077] Obtain the gradient magnitudes of w_r and w_t'', and calculate their similarity.
[0078] 4) Overall similarity Calculate the FSIM value by combining phase consistency and gradient magnitude:
[0079] The final result is usually normalized to the range of [0, 1], and the closer the value is to 1, the better the image quality.
[0080] It should also be noted that the image quality index UQI and feature similarity index FSIM are used as quantitative analysis results, and the factory parameters of the digital pathology slide scanner to be verified are adjusted based on the thermal analysis results and the quantitative analysis results.
[0081] In this embodiment, the WSI image of the pathological slide to be tested is first acquired using the digital pathology slide scanner to be calibrated, and a reference WSI image of the pathological slide is obtained. Then, the reference WSI image and the WSI image to be tested are coarsely registered to obtain a coarsely registered WSI image. Foreground segmentation is then performed on both the reference WSI image and the coarsely registered WSI image to obtain a reference foreground region image and a coarsely registered foreground region image, respectively. Next, the reference foreground region image and the coarsely registered foreground region image are finely registered to obtain a finely registered WSI image. Finally, thermal analysis is performed on the finely registered WSI image, and the factory parameters of the digital pathology slide scanner to be calibrated are adjusted based on the thermal analysis results. Compared to the prior art where engineers need to manually check the imaging quality of the equipment and adjust parameters by comparing the differences between the digital image and the microscopic image, resulting in low efficiency in adjusting the scanner's factory parameters, this invention performs coarse and fine registration between the WSI image to be tested and the reference WSI image, and then performs thermal analysis on the finely registered WSI image. This provides an intuitive thermal map to assist engineers in adjusting the equipment, improving the efficiency of adjusting the factory parameters of the digital pathology slide scanner and reducing equipment production costs.
[0082] Reference Figure 3 , Figure 3 This is a structural block diagram of the first embodiment of the debugging system for the factory parameters of the digital pathology slide scanner of the present invention.
[0083] like Figure 3 As shown, the debugging system for the factory parameters of the digital pathology slide scanner proposed in this embodiment of the invention includes: The acquisition module 3001 is used to acquire the WSI image of the pathological slide to be examined through the digital pathological slide scanner to be verified, and to obtain the reference WSI image of the pathological slide through the standard digital pathological slide scanner. Registration module 3002 is used to coarsely register the reference WSI image and the WSI image to be inspected to obtain a coarsely registered WSI image. The segmentation module 3003 is used to perform foreground segmentation on the reference WSI image and the coarse registration WSI image respectively to obtain a reference foreground region image and a coarse registration foreground region image; The registration module 3002 is further configured to perform fine registration on the reference foreground region image and the coarsely registered foreground region image to obtain a finely registered WSI image; The debugging module 3004 is used to perform thermal analysis on the finely registered WSI image and to debug the factory parameters of the digital pathology slide scanner to be verified based on the thermal analysis results.
[0084] Other embodiments or specific implementations of the debugging system for the factory parameters of the digital pathology slide scanner of the present invention can be referred to the above-described method embodiments, and will not be repeated here.
[0085] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0086] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0088] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.
Claims
1. A method for adjusting the factory parameters of a digital pathology slide scanner, characterized in that, The method includes the following steps: The WSI image of the pathological slide to be examined is acquired by a digital pathological slide scanner to be verified, and a reference WSI image of the pathological slide is obtained by a standard digital pathological slide scanner. The reference WSI image and the WSI image to be inspected are coarsely registered to obtain a coarsely registered WSI image. Foreground segmentation is performed on the reference WSI image and the coarse registration WSI image respectively to obtain a reference foreground region image and a coarse registration foreground region image; The reference foreground region image and the coarsely registered foreground region image are finely registered to obtain a finely registered WSI image; Thermal analysis was performed on the precisely registered WSI image, and the factory parameters of the digital pathology slide scanner to be verified were adjusted based on the thermal analysis results.
2. The method as described in claim 1, characterized in that, The step of coarsely registering the reference WSI image and the WSI image to be detected to obtain a coarsely registered WSI image includes: SIFT key points corresponding to the reference WSI image and the WSI image to be inspected are extracted using a SIFT detector. Select coarse matching feature point pairs from multiple SIFT key points in the reference WSI image and multiple SIFT key points in the WSI image to be inspected; Calculate the coarse registration transformation matrix based on the coarse matching feature point pairs; Based on the coarse registration transformation matrix, the WSI image to be detected is mapped to the coordinates of the reference WSI image to obtain the coarse registration WSI image.
3. The method as described in claim 2, characterized in that, The step of selecting coarse matching feature point pairs from multiple SIFT keypoints in the reference WSI image and multiple SIFT keypoints in the WSI image to be inspected includes: The SIFT detector extracts the descriptors corresponding to each SIFT keypoint in the reference WSI image and the descriptors corresponding to each SIFT keypoint in the WSI image to be tested. Calculate the distance between the descriptors of each SIFT keypoint in the reference WSI image and the descriptors of each SIFT keypoint in the WSI image to be inspected; Based on the distance, coarse matching feature point pairs are selected from multiple SIFT keypoints in the reference WSI image and multiple SIFT keypoints in the WSI image to be inspected.
4. The method as described in claim 1, characterized in that, The step of performing foreground segmentation on the reference WSI image and the coarsely registered WSI image respectively to obtain a reference foreground region image and a coarsely registered foreground region image includes: Multiple WSI image samples were obtained, and contour annotations were performed on each WSI image sample to obtain multiple WSI image annotation samples and the corresponding masks for each WSI image annotation sample. A tissue segmentation training set is constructed based on each WSI image annotation sample and the corresponding mask for each WSI image annotation sample. An initial image segmentation model is trained using the tissue segmentation training set, and the initial image segmentation model is built based on the deeplabv3+ framework; The reference WSI image and the coarse-registered WSI image are respectively input into the trained image segmentation model, and the foreground mask of the reference WSI image and the foreground mask of the coarse-registered WSI image are output. A reference foreground region image is obtained based on the reference WSI image and the foreground mask of the reference WSI image, and a coarse registration foreground region image is obtained based on the coarse registration WSI image and the foreground mask of the coarse registration WSI image.
5. The method as described in claim 1, characterized in that, The step of performing fine registration between the reference foreground region image and the coarsely registered foreground region image to obtain a finely registered WSI image includes: The reference foreground region image and the coarsely registered foreground region image are respectively divided into image blocks; Calculate the distance between the descriptors of multiple SIFT keypoints in each reference foreground region image block and the descriptors of multiple SIFT keypoints in each coarsely registered foreground region image block; Based on the distance, fine-matching feature point pairs are selected from multiple SIFT keypoints within each reference foreground region image block and multiple SIFT keypoints within each coarse-registered foreground region image block. Calculate the fine registration transformation matrix based on the fine-matched feature point pairs; Based on the fine registration transformation matrix, the coarse registration foreground region image is mapped to the coordinates of the reference foreground region image to obtain the fine registration WSI image.
6. The method as described in claim 5, characterized in that, The step of calculating the fine registration transformation matrix based on the finely matched feature point pairs includes: Construct a linear equation for the matching point pair based on the coordinates and homography matrix of the finely matched feature point pair; The fine registration transformation matrix is obtained by solving the linear equation of the matching point pair using the least squares method.
7. The method as described in claim 1, characterized in that, The thermal analysis of the finely registered WSI image includes: Extract the three-channel reference grayscale image and the three-channel finely registered grayscale image corresponding to the reference foreground region image and the finely registered WSI image, respectively, and calculate the three-channel grayscale difference based on the three-channel reference grayscale image and the three-channel finely registered grayscale image; Calculate the gradients in different directions corresponding to the reference foreground region image and the finely registered WSI image respectively, and calculate the reference intensity matrix and the finely registered intensity matrix corresponding to the reference foreground region image and the finely registered WSI image respectively based on the gradients in different directions; The intensity difference between each pixel in the reference foreground region image and each pixel in the finely registered WSI image is calculated based on the reference intensity matrix and the finely registered intensity matrix. The pixels corresponding to the three-channel grayscale difference and the pixels corresponding to the intensity difference are mapped to a preset color space to obtain a three-channel grayscale difference heatmap and a pixel intensity difference heatmap. The three-channel grayscale difference heatmap and the pixel intensity difference heatmap are superimposed on the WSI image to be inspected for thermal analysis.
8. The method as described in claim 1, characterized in that, After performing thermal analysis on the finely registered WSI image, the process further includes: Quantitative analysis is performed on the reference foreground region image and the finely registered WSI image; The factory parameters of the digital pathology slide scanner to be verified were adjusted based on the results of thermal analysis and the quantitative analysis.
9. A system for adjusting the factory parameters of a digital pathology slide scanner, characterized in that, The system includes: The acquisition module is used to acquire the WSI image of the pathological slide to be examined through the digital pathological slide scanner to be verified, and to obtain the reference WSI image of the pathological slide. The registration module is used to coarsely register the reference WSI image and the WSI image to be inspected to obtain a coarsely registered WSI image. The segmentation module is used to perform foreground segmentation on the reference WSI image and the coarse registration WSI image respectively, to obtain a reference foreground region image and a coarse registration foreground region image; The registration module is further configured to perform fine registration on the reference foreground region image and the coarsely registered foreground region image to obtain a finely registered WSI image; The debugging module is used to perform thermal analysis on the finely registered WSI image and to debug the factory parameters of the digital pathology slide scanner to be verified based on the thermal analysis results.
10. A medium, characterized in that, The storage medium stores a debugging program for the factory parameters of the digital pathology slide scanner. When the debugging program for the factory parameters of the digital pathology slide scanner is executed by the processor, it implements the steps of the debugging method for the factory parameters of the digital pathology slide scanner as described in any one of claims 1 to 8.