Digital pathology image high-color fidelity splicing method and device based on feature point matching

Abstract

The application provides a digital pathological image high-color-precision splicing method and device based on feature point matching, which comprises the following steps: obtaining a target splicing area in a target splicing image and a to-be-spliced area in a to-be-spliced image based on the scanning sequence of the digital pathological image; performing affine transformation on the to-be-spliced area by using an affine transformation matrix to obtain an initial correction area; constructing a global color mapping function, and mapping the initial correction image by using the global color mapping function to obtain a color correction image; and splicing the color correction image and the target splicing image based on the scanning sequence of the pathological image to obtain a digital pathological image splicing result. According to the scheme, feature point matching is only performed on the target splicing area and the to-be-spliced area, the cumulative distribution function of the effective feature points is calculated respectively according to RGB channels, and an optimized global color mapping function is constructed, so that accurate color correction is realized, and the visual integrity and consistency of the splicing image are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing, in particular to a digital pathology image high-color-accuracy splicing method and device based on feature point matching. BACKGROUND

[0002] As the core high-throughput equipment of the digital transformation of pathological diagnosis, the digital pathology scanner converts traditional pathological sections into whole section digital images (Whole Slide Imaging, WSI) through optical scanning technology, providing a basic support for the storage, remote transmission and AI assisted analysis of pathological images. Its technical research and development involves multiple disciplines such as optical design, precision mechanical control, image algorithm and software integration, and the technical barriers are extremely high. In the clinical diagnosis scene, the recognition and judgment of pathological changes by pathologists are highly dependent on the color features of the stained tissue sections, such as the staining depth of cell structure and the color difference of different tissue regions, etc. Therefore, the color restoration degree of digital pathology images directly determines the diagnostic accuracy. Once there is color deviation, it may lead to misjudgment of key pathological information. At the same time, the color output difference of different laboratories and different types of scanners also seriously hinders the standardization of multi-center collaborative research and remote consultation.

[0003] However, in the high-speed scanning process of the digital pathology scanner, a series of hardware and environmental factors can easily cause image quality problems: the dynamic movement of the scanning platform is not stable, which can cause image misalignment; the fluctuation of focusing accuracy, the attenuation of light source brightness with scanning time, the slight drift of camera parameters (such as exposure, white balance), and the difference in radiation characteristics of the imaging system can all cause significant color inconsistency in the multiple images to be spliced taken in time sequence. These problems directly reflect in the image splicing link, and the complete WSI after splicing has obvious splicing seams, edge artifacts, and significant color difference between different regions, which not only destroys the visual integrity of the image, but also may interfere with the reliability of subsequent pathological analysis.

[0004] To address these issues, two core technological directions have emerged in the industry: First, image alignment technology, including pixel-based alignment (optimizing parameters by moving the image and comparing pixel differences, which is inefficient and susceptible to noise) and feature-based alignment (such as scale-invariant feature detection combined with least squares estimation of geometric transformations, but the computational load for full-image feature extraction and matching is extremely high); second, color transfer technology, one type based on geometric transformations (relying on image feature point matching to construct a color mapping function, but unusable if feature detection fails), and the other based on statistical correlation (constructing a mapping through global pixel statistics, which, while not relying on feature points, results in poor real-time performance due to full-image pixel processing). Crucially, digital pathology scanning has an inherent technical characteristic—the overlapping areas of the images to be stitched are determined by shooting along a preset path, with overlap accuracy reaching the micrometer level. Existing technologies do not fully utilize this characteristic, still employing a full-image processing mode, leading to high computational loads for feature point retrieval and color calculation, making it difficult to meet the real-time stitching requirements of high-speed scanning scenarios and failing to balance image quality and processing efficiency. Against this backdrop, the development of a technical solution that can efficiently eliminate stitching seams and artifacts and achieve color correction by relying on preset overlapping areas has become an urgent need in the field of digital pathology scanning. Summary of the Invention

[0005] This application provides a method and apparatus for high color accuracy stitching of digital pathological images based on feature point matching. Feature point matching is performed only on the target stitching area and the area to be stitched. The cumulative distribution function of effective feature points is calculated for each RGB channel, and an optimized global color mapping function is constructed to achieve accurate color correction, reduce color difference between images, ensure the color reproduction accuracy that is relied upon for pathological diagnosis, and improve the visual integrity and consistency of the stitched images.

[0006] In a first aspect, embodiments of this application provide a method for high-color-accuracy stitching of digital pathological images based on feature point matching, the method comprising:

[0007] Based on the scanning order of digital pathological images, the target splicing image and the image to be spliced ​​are obtained in temporal adjacent order. Based on the preset splicing region size, the target splicing region in the target splicing image and the region to be spliced ​​in the image to be spliced ​​are obtained.

[0008] Affine transformation matrix is ​​obtained by matching feature points of the region to be stitched and the target region. The affine transformation matrix is ​​then used to perform an affine transformation on the region to be stitched to obtain an initial correction region. Feature points that are correctly matched between the initial correction region and the target region are taken as valid feature points.

[0009] The target function set is obtained by calculating the target cumulative distribution function of the effective feature points in each RGB channel within the target stitching region. The initial correction function set is obtained by calculating the initial correction cumulative distribution function of the effective feature points in each RGB channel within the initial correction region. A global color mapping function is constructed based on the initial correction function set and the target function set. An affine transformation is performed on the image to be stitched using an affine transformation matrix to obtain an initial corrected image. The initial corrected image is then mapped using the global color mapping function to obtain a color corrected image.

[0010] The color-corrected image is stitched together with the target stitched image based on the scanning order of the pathological images to obtain the digital pathological image stitching result.

[0011] Secondly, embodiments of this application provide a digital pathological image high-color-accuracy stitching device based on feature point matching, comprising:

[0012] The acquisition module acquires temporally adjacent target splicing images and images to be spliced ​​based on the scanning order of digital pathological images, and acquires the target splicing region in the target splicing image and the region to be spliced ​​in the image to be spliced ​​based on the preset splicing region size.

[0013] The feature point matching module is used to perform feature point matching between the region to be stitched and the target region to obtain an affine transformation matrix. The affine transformation matrix is ​​then used to perform an affine transformation on the region to be stitched to obtain an initial correction region. Feature points that are correctly matched between the initial correction region and the target region to be stitched are taken as valid feature points.

[0014] The color correction module is used to calculate the target cumulative distribution function of the effective feature points in each RGB channel within the target stitching area to obtain the target function set, calculate the initial correction cumulative distribution function of the effective feature points in each RGB channel within the initial correction area to obtain the initial correction function set, construct a global color mapping function based on the initial correction function set and the target function set, perform an affine transformation on the image to be stitched using an affine transformation matrix to obtain the initial corrected image, and map the initial corrected image using the global color mapping function to obtain the color corrected image;

[0015] The stitching module stitches the color-corrected image with the target stitching image based on the scanning order of the pathological images to obtain the digital pathological image stitching result.

[0016] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform a high-color-accuracy stitching method for digital pathological images based on feature point matching.

[0017] Fourthly, embodiments of this application provide a readable storage medium storing a computer program that, when executed by a processor, implements a high-color-accuracy stitching method for digital pathological images based on feature point matching.

[0018] The main contributions and innovations of this invention are as follows:

[0019] This application embodiment only performs feature point matching on the target stitching area and the area to be stitched, avoiding full-image computation, significantly reducing the computational load, and meeting the real-time stitching requirements of high-speed scanning scenarios. This application embodiment uses KD-tree to construct feature index + bidirectional KNN matching + RANSAC algorithm to remove mismatched points, improving the accuracy and efficiency of feature point matching, ensuring the reliability of the affine transformation matrix, and reducing image stitching misalignment. This application embodiment calculates the cumulative distribution function of effective feature points for each RGB channel and constructs an optimized global color mapping function to achieve accurate color correction, reduce color differences between images, and ensure the color reproduction accuracy relied upon for pathological diagnosis. This application embodiment uses a dynamic programming algorithm to obtain the optimal stitching seam with the minimum sum of pixel energy values ​​and merges them, eliminating stitching seams and edge artifacts, and improving the visual integrity and consistency of the stitched image.

[0020] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 This is a flowchart of a high-color-accuracy stitching method for digital pathological images based on feature point matching, according to an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of a target image to be stitched and an image to be stitched, according to an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of a target splicing area and a splicing area according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of a final matching result according to an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of a digital pathological image stitching result according to an embodiment of this application;

[0027] Figure 6 This is a structural block diagram of a digital pathology image high color accuracy stitching device based on feature point matching according to an embodiment of this application;

[0028] Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0030] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0031] Example 1

[0032] This application provides a high-color-accuracy stitching method for digital pathological images based on feature point matching. This method only performs feature point matching on the target stitching region and the region to be stitched, and calculates the cumulative distribution function of effective feature points for each RGB channel, constructing an optimized global color mapping function to achieve accurate color correction, reduce color differences between images, ensure color reproduction crucial for pathological diagnosis, and improve the visual integrity and consistency of the stitched images. Specifically, refer to... Figure 1 The method includes:

[0033] Based on the scanning order of digital pathological images, the target splicing image and the image to be spliced ​​are obtained in temporal adjacent order. Based on the preset splicing region size, the target splicing region in the target splicing image and the region to be spliced ​​in the image to be spliced ​​are obtained.

[0034] Affine transformation matrix is ​​obtained by matching feature points of the region to be stitched and the target region. The affine transformation matrix is ​​then used to perform an affine transformation on the region to be stitched to obtain an initial correction region. Feature points that are correctly matched between the initial correction region and the target region are taken as valid feature points.

[0035] The target function set is obtained by calculating the target cumulative distribution function of the effective feature points in each RGB channel within the target stitching region. The initial correction function set is obtained by calculating the initial correction cumulative distribution function of the effective feature points in each RGB channel within the initial correction region. A global color mapping function is constructed based on the initial correction function set and the target function set. An affine transformation is performed on the image to be stitched using an affine transformation matrix to obtain an initial corrected image. The initial corrected image is then mapped using the global color mapping function to obtain a color corrected image.

[0036] The color-corrected image is stitched together with the target stitched image based on the scanning order of the pathological images to obtain the digital pathological image stitching result.

[0037] High-resolution digital pathology images are composed of stitched-together sub-images. During the stitching process, a pre-defined stitching region size is set, representing the overlapping area between two sequentially adjacent sub-images. In this solution, the stitching region size is set to 10%. This means that for a temporally adjacent target image and image to be stitched, the right 10% of the target image and the left 10% of the image to be stitched constitute the overlapping area. For example, the target image and the image to be stitched are as follows: Figure 2 As shown, in Figure 2 As can be seen, the right side of the target image to be stitched and the left side of the image to be stitched are partially identical.

[0038] Therefore, in the current embodiment, the target stitching region and the region to be stitched are first obtained based on the predicted stitching region size. The target stitching region and the region to be stitched are as follows: Figure 3 As shown, Figure 3 The bounding box in the image represents both the target stitching area and the area to be stitched. By processing the area to be stitched, the image to be stitched is stitched together with the target image to complete the acquisition of the digital pathology image.

[0039] In the current embodiment, feature points are extracted from the target splicing region and the region to be spliced ​​to obtain a target splicing feature point set and a region to be spliced ​​to obtain a feature point set. KNN matching is then performed on the target splicing feature point set and the region to be spliced ​​to obtain an initial feature point matching result. Mismatched feature point pairs in the initial matching result are removed to obtain a final matching result. An affine transformation matrix is ​​constructed based on the final matching result, and the feature points in the final matching result are considered valid feature points.

[0040] Specifically, the SURF algorithm is used to extract feature points from both the target splicing region and the region to be spliced. The SURF algorithm mainly includes:

[0041] 1. An integral image is used to represent the target stitching region and the region to be stitched. The sum of the gray values ​​of the target stitching region and the region to be stitched is obtained by addition and subtraction within the integral image. Processing the target stitching region and the region to be stitched by addition and subtraction can reduce the computation time. Compared with processing the original images such as the target stitching image and the image to be stitched, this scheme has higher computational efficiency.

[0042] 2. By changing the size of the filter, a multi-scale space is constructed. Within the multi-scale space, the target splicing region and the region to be spliced ​​represented by the integral image are processed to obtain feature points of different scales.

[0043] 3. Calculate the determinant of the Hessian matrix. By comparing the magnitudes of the determinants of the Hessian matrix, determine the location and scale of the feature points, thereby identifying local extreme feature points.

[0044] 4. Haar wavelets are used as feature descriptors to obtain the Haar wavelet response of the region surrounding each feature point, thereby generating the feature description vector corresponding to each feature point.

[0045] Furthermore, a KD-tree is constructed based on the feature description vector of each feature point in the target splicing feature point set to obtain a target splicing tree diagram. A KD-tree is also constructed based on the feature description vector of each feature point in the set to be spliced ​​to obtain a tree diagram to be spliced. The matching result between the target splicing tree diagram and each feature point in the tree diagram to be spliced ​​is obtained based on the KNN matching algorithm as the first matching result. The matching result between the tree diagram to be spliced ​​and each feature point in the target splicing tree diagram is obtained based on the KNN matching algorithm as the second matching result. The feature point pairs that match in the first matching result and the second matching result are integrated as the initial matching result of the feature points.

[0046] Specifically, when constructing the target stitching tree diagram, the feature description vector of each feature point in the target stitching feature point set is constructed on a KD-tree to establish a feature data index. Then, the segmentation domain and feature nodes are determined. The feature point position coordinates are two-dimensional data points. The variances in the X and Y dimensions are calculated separately. The dimension with the largest variance is taken as the currently obtained segmentation domain and segmented. The feature point coordinate values ​​are sorted according to the dimension with the largest variance. The median is selected as the current node, and this node is used to segment the spatial domain, resulting in left and right subspaces. Coordinates less than the node value belong to the left space, and coordinates greater than the node value belong to the right space. Then, the coordinates contained in the subspaces are iteratively divided until the feature point data in the target stitching feature point set is empty, thus obtaining the target stitching tree diagram. The construction method of the tree diagram to be stitched is the same as that of the target stitching tree diagram and will not be repeated.

[0047] Specifically, in obtaining the first matching result, the KNN matching algorithm is used to obtain the top K feature points in the target stitched tree diagram that have the highest similarity to each feature point in the tree diagram to be stitched as the first matching result. Similarly, in obtaining the second matching result, the KNN matching algorithm is used to obtain the top K feature points in the target stitched tree diagram that have the highest similarity to each feature point in the tree diagram to be stitched as the second matching result. The feature point pairs that match in the first matching result and the second matching result are integrated as the initial matching result of the feature points. That is, when a pair of feature points has the highest similarity to each other in the first matching result and the second matching result, the corresponding feature point pair is used as the matching feature point in the initial matching result.

[0048] Specifically, the RANSAC algorithm is used to remove mismatched pairs from the initial matching results to obtain the final matching results. A schematic diagram of the final matching results is shown below. Figure 4 As shown, the RANSAC algorithm includes:

[0049] Step 1: Randomly obtain 3 feature point pairs from the initial matching results, and estimate the parameters of the initial matrix model H based on the 3 feature point pairs;

[0050] Step 2: Calculate the distance between the feature points in the initial matching result that belong to the target splicing feature point set and the corresponding matching feature points after transformation by the matrix model H. If the distance is less than the set threshold, the point is an interior point; otherwise, it is a mismatch point.

[0051] Step 3: Repeat steps 1 and 2 n times, calculating the number of interior points each time. After processing all feature points, stop the iteration, take the matrix H corresponding to the maximum number of interior points, and select the set with the most interior points as the interior point set. Use the feature point pairs corresponding to the interior point set as the final matching result, thus retaining correct matches and eliminating incorrect matches.

[0052] Furthermore, an affine transformation matrix is ​​constructed based on the matched feature point pairs in the final matching result. The affine transformation matrix is ​​used to perform an affine transformation on the region to be stitched to obtain an initial correction region. The coordinates of each pixel in the initial correction region are the same as the coordinates of the corresponding pixel in the target stitching region.

[0053] Specifically, an affine transformation matrix is ​​used to perform an affine transformation on the area to be stitched, thereby aligning the pixels of the area to be stitched with the corresponding pixels of the target area to avoid pixel misalignment during subsequent stitching.

[0054] In the current embodiment, the target stitching region is separated into RGB channels, and the target histogram data distribution of effective feature points under each RGB channel is calculated. Based on the target histogram data distribution of effective feature points under each RGB channel, the target cumulative distribution function of effective feature points under each RGB channel is calculated to obtain the target function set. The initial correction region is separated into RGB channels, and the initial correction histogram data distribution of effective feature points under each RGB channel is calculated. Based on the initial correction histogram data distribution of effective feature points under each RGB channel, the initial correction cumulative distribution function of effective feature points under each RGB channel is calculated to obtain the initial correction function set.

[0055] Specifically, the target stitching region is separated into a target R-channel image, a target G-channel image, and a target B-channel image. The R-color values ​​of all effective feature points in the target R-channel image are statistically analyzed to obtain the target histogram data distribution of the corresponding target R-channel image; the G-color values ​​of all effective feature points in the target G-channel image are statistically analyzed to obtain the target histogram data distribution of the corresponding target G-channel image; and the B-color values ​​of all effective feature points in the target B-channel image are statistically analyzed to obtain the target histogram data distribution of the corresponding target B-channel image. Then, based on the target histogram data distribution of the target R channel image, the proportion of different R color values ​​appearing in all effective feature points is statistically analyzed to obtain the target cumulative distribution function under the target R channel image; then, based on the target histogram data distribution of the target G channel image, the proportion of different G color values ​​appearing in all effective feature points is statistically analyzed to obtain the target cumulative distribution function under the target G channel image; then, based on the target histogram data distribution of the target B channel image, the proportion of different B color values ​​appearing in all effective feature points is statistically analyzed to obtain the target cumulative distribution function under the target B channel image. Integrating the target cumulative distribution functions under the target R channel image, the target cumulative distribution functions under the target G channel image, and the target cumulative distribution functions under the target B channel image yields the target function set.

[0056] Similarly, the initial calibration region is separated into an initial calibration R-channel image, an initial calibration G-channel image, and an initial calibration B-channel image. The R-color values ​​of all effective feature points in the initial calibration R-channel image are statistically analyzed to obtain the initial calibration histogram data distribution of the corresponding initial calibration R-channel image; the G-color values ​​of all effective feature points in the initial calibration G-channel image are statistically analyzed to obtain the initial calibration histogram data distribution of the corresponding initial calibration G-channel image; and the B-color values ​​of all effective feature points in the initial calibration B-channel image are statistically analyzed to obtain the initial calibration histogram data distribution of the corresponding initial calibration B-channel image. Then, based on the initial calibration histogram data distribution of the initial calibration R channel image, the proportion of different R color values ​​appearing in all effective feature points is statistically analyzed to obtain the initial calibration cumulative distribution function under the initial calibration R channel image; then, based on the initial calibration histogram data distribution of the initial calibration G channel image, the proportion of different G color values ​​appearing in all effective feature points is statistically analyzed to obtain the initial calibration cumulative distribution function under the initial calibration G channel image; then, based on the initial calibration histogram data distribution of the initial calibration B channel image, the proportion of different B color values ​​appearing in all effective feature points is statistically analyzed to obtain the initial calibration cumulative distribution function under the initial calibration B channel image. Integrating the target cumulative distribution functions under the initial calibration R channel image, the initial calibration G channel image, and the initial calibration B channel image yields the target function set.

[0057] Specifically, the histogram data distribution represents the number of times each pixel color value appears, and the cumulative distribution function represents the proportion of each pixel color value that appears.

[0058] Furthermore, based on the objective function set and the initial correction function set, histogram normalization is performed on the initial correction region to obtain the first color correction region. With the goal of minimizing the color difference between the first color correction region and the initial correction region, the color mapping function for each RGB channel is calculated to obtain the initial color mapping function. The formula for calculating the color mapping function for each RGB channel is expressed as follows:

[0059]

[0060] Wherein, the grayscale value of a k-pixel... This is the mapping function for the pixel grayscale value k in the RGB channel. The coordinates of the valid feature points are numbered. , Let k be the set of valid feature points with pixel color value k in the initial correction region. This is the initial correction region. Valid feature points in the initial correction region pixel color values, This is the first color correction area. Valid feature points in the first color correction region The pixel color value is denoted as c, and the corresponding pixel color value is mapped to c.

[0061] The initial color mapping function is applied to each valid feature point within the initial correction region to obtain the second color correction region. The color difference between the valid feature points in the second color correction region and the corresponding valid feature points in the first color correction region in each RGB channel is calculated. Valid feature points with color differences greater than a set threshold are removed to obtain the updated color correction region. Use the updated version The global color mapping function is obtained by recalculating the color mapping function for each RGB channel.

[0062] Specifically, the formula for applying the initial color mapping function to each effective feature point within the initial correction region is expressed as:

[0063]

[0064] Where MAP is the initial color mapping function. Valid feature points within the initial correction region pixel color values, Effective feature points within the second color correction region The pixel color values.

[0065] Specifically, the formula for calculating the color difference between the effective feature points of the second color correction region and the corresponding effective feature points of the first color correction region in each RGB channel is expressed as:

[0066]

[0067] in, Valid feature points The color difference, Effective feature points within the second color correction region pixel color values, Valid feature points within the first color correction region The pixel color values.

[0068] Specifically, this scheme first performs an affine transformation on the images to be stitched using an affine transformation matrix to obtain an initial corrected image. Then, it constructs a global color mapping function for each RGB channel. By processing the initial corrected image using the global mapping function, the colors of the initial corrected image and the target stitched image are made consistent, thereby achieving a high color accuracy stitching effect in subsequent stitching.

[0069] In the current embodiment, the overlapping area for stitching the color-corrected image and the target stitched image is obtained based on a preset stitching area size. The pixel grayscale difference between two corresponding overlapping pixels in the overlapping area is used as the pixel energy value of the corresponding pixel. Based on a dynamic programming algorithm, the optimal stitching seam is obtained with the constraint that the stitching seam in the overlapping area is continuous and the sum of the pixel energy values ​​in the stitching seam is minimized. The color-corrected image and the target stitched image are stitched together based on the optimal stitching seam.

[0070] For example, when obtaining the optimal seam based on a dynamic programming algorithm, any pixel in the first row of the overlapping area is taken as the starting point P, which is then used as the first pixel of the seam. After obtaining the energy value of the starting point P, the search continues to the next row. The search direction is the pixels directly below, 45 degrees to the lower left, and 45 degrees to the lower right of the starting point P. The search method is to calculate the sum of the energy values ​​of the starting point P and the three pixels in the next row, and take the pixel P' with the smallest sum of energy values ​​as the next search point, and set P=P', until the current search point is a pixel in the last row. Through multiple rounds of iterative search, the seam with the smallest sum of all pixel energy values ​​is selected as the optimal seam. Specifically, this scheme can choose any dynamic programming algorithm to obtain the optimal seam, and this scheme does not restrict the specific dynamic programming algorithm.

[0071] In this scheme, any stitching method can be used to stitch the color-corrected image and the target stitched image in order to obtain the optimal stitching seam. This scheme does not restrict the specific stitching method.

[0072] In some embodiments, the Laplacian image fusion algorithm is used to stitch images based on the obtained optimal stitching seams. After stitching, the optimal stitching seams are smoothed to obtain a complete stitched image. The digital pathology image stitching result in this scheme is as follows: Figure 5 As shown.

[0073] For example, the CIEDE2000 color standard was used to evaluate color difference. Before stitching the target image with the image to be stitched, the average color difference between the target image and the image to be stitched was 19.97, while the average color difference in the stitched digital pathology image was 9.83. The color accuracy of the stitched image was greatly improved, and the effects of stitching seams and artifacts were eliminated.

[0074] Example 2

[0075] Based on the same concept, referencing Figure 6 This application also proposes a high-color-accuracy stitching device for digital pathology images based on feature point matching, comprising:

[0076] The acquisition module acquires temporally adjacent target splicing images and images to be spliced ​​based on the scanning order of digital pathological images, and acquires the target splicing region in the target splicing image and the region to be spliced ​​in the image to be spliced ​​based on the preset splicing region size.

[0077] The feature point matching module is used to perform feature point matching between the region to be stitched and the target region to obtain an affine transformation matrix. The affine transformation matrix is ​​then used to perform an affine transformation on the region to be stitched to obtain an initial correction region. Feature points that are correctly matched between the initial correction region and the target region to be stitched are taken as valid feature points.

[0078] The color correction module is used to calculate the target cumulative distribution function of the effective feature points in each RGB channel within the target stitching area to obtain the target function set, calculate the initial correction cumulative distribution function of the effective feature points in each RGB channel within the initial correction area to obtain the initial correction function set, construct a global color mapping function based on the initial correction function set and the target function set, perform an affine transformation on the image to be stitched using an affine transformation matrix to obtain the initial corrected image, and map the initial corrected image using the global color mapping function to obtain the color corrected image;

[0079] The stitching module stitches the color-corrected image with the target stitching image based on the scanning order of the pathological images to obtain the digital pathological image stitching result.

[0080] Example 3

[0081] This embodiment also provides an electronic device, see reference. Figure 7 It includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0082] Specifically, the processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0083] Memory 404 may include a mass storage device for data or instructions. For example, and not limitingly, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to a data processing device. In a particular embodiment, memory 404 is non-volatile memory. In a particular embodiment, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0084] The memory 404 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 402.

[0085] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any of the high color accuracy stitching methods for digital pathological images based on feature point matching in the above embodiments.

[0086] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408, wherein the transmission device 406 is connected to the processor 402, and the input / output device 408 is connected to the processor 402.

[0087] The transmission device 406 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 406 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0088] The input / output device 408 is used to input or output information. In this embodiment, the input information may be the target image to be stitched, the image to be stitched, etc., and the output information may be the digital pathology image stitching result, etc.

[0089] Optionally, in this embodiment, the processor 402 can be configured to perform the following steps via a computer program:

[0090] Based on the scanning order of digital pathological images, the target splicing image and the image to be spliced ​​are obtained in temporal adjacent order. Based on the preset splicing region size, the target splicing region in the target splicing image and the region to be spliced ​​in the image to be spliced ​​are obtained.

[0091] Affine transformation matrix is ​​obtained by matching feature points of the region to be stitched and the target region. The affine transformation matrix is ​​then used to perform an affine transformation on the region to be stitched to obtain an initial correction region. Feature points that are correctly matched between the initial correction region and the target region are taken as valid feature points.

[0092] The target function set is obtained by calculating the target cumulative distribution function of the effective feature points in each RGB channel within the target stitching region. The initial correction function set is obtained by calculating the initial correction cumulative distribution function of the effective feature points in each RGB channel within the initial correction region. A global color mapping function is constructed based on the initial correction function set and the target function set. An affine transformation is performed on the image to be stitched using an affine transformation matrix to obtain an initial corrected image. The initial corrected image is then mapped using the global color mapping function to obtain a color corrected image.

[0093] The color-corrected image is stitched together with the target stitched image based on the scanning order of the pathological images to obtain the digital pathological image stitching result.

[0094] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0095] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0096] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets, and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product may include one or more computer-executable components configured to perform the embodiments when the program is run. The one or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted in this respect that, as Figure 7 Any box in the logical flow can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0097] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0098] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A high-color-accuracy stitching method for digital pathological images based on feature point matching, characterized in that, Includes the following steps: Based on the scanning order of digital pathological images, the target splicing image and the image to be spliced ​​are obtained in temporal adjacent order. Based on the preset splicing region size, the target splicing region in the target splicing image and the region to be spliced ​​in the image to be spliced ​​are obtained. Affine transformation matrix is ​​obtained by matching feature points of the region to be stitched and the target region. The affine transformation matrix is ​​then used to perform an affine transformation on the region to be stitched to obtain an initial correction region. Feature points that are correctly matched between the initial correction region and the target region are taken as valid feature points. The target function set is obtained by calculating the target cumulative distribution function of the effective feature points in each RGB channel within the target stitching region. The initial correction function set is obtained by calculating the initial correction cumulative distribution function of the effective feature points in each RGB channel within the initial correction region. A global color mapping function is constructed based on the initial correction function set and the target function set. An affine transformation is performed on the image to be stitched using an affine transformation matrix to obtain an initial corrected image. The initial corrected image is then mapped using the global color mapping function to obtain a color corrected image. The color-corrected image and the target stitched image are stitched together based on the scanning order of the pathological images to obtain the digital pathological image stitching result. Specifically, the overlapping area between the color-corrected image and the target stitched image is obtained based on a preset stitching area size. The pixel grayscale difference between two corresponding overlapping pixels in the overlapping area is used as the pixel energy value of the corresponding pixel. Based on a dynamic programming algorithm, the optimal stitching seam is obtained with the constraint that the stitching seam in the overlapping area is continuous and the sum of the pixel energy values ​​in the stitching seam is minimized. The color-corrected image and the target stitched image are then stitched together based on the optimal stitching seam.

2. The method for high color accuracy stitching of digital pathological images based on feature point matching according to claim 1, characterized in that, Feature points are extracted from the target splicing region and the region to be spliced ​​to obtain the target splicing feature point set and the region to be spliced ​​to obtain the feature point set. KNN matching is then performed on the target splicing feature point set and the region to be spliced ​​to obtain the initial feature point matching result. Mismatched feature point pairs in the initial matching result are removed to obtain the final matching result. An affine transformation matrix is ​​constructed based on the final matching result. The feature points in the final matching result are the valid feature points.

3. The method for high color accuracy stitching of digital pathological images based on feature point matching according to claim 2, characterized in that, A KD-tree is constructed based on the feature description vector of each feature point in the target splicing feature point set to obtain a target splicing tree diagram. A KD-tree is also constructed based on the feature description vector of each feature point in the set to be spliced ​​to obtain a tree diagram to be spliced. The matching result between the target splicing tree diagram and each feature point in the tree diagram to be spliced ​​is obtained based on the KNN matching algorithm as the first matching result. The matching result between the tree diagram to be spliced ​​and each feature point in the target splicing tree diagram is obtained based on the KNN matching algorithm as the second matching result. The feature point pairs that match in the first matching result and the second matching result are integrated as the initial matching result of the feature points.

4. The method for high color accuracy stitching of digital pathological images based on feature point matching according to claim 1, characterized in that, The target stitching region is separated into RGB channels, and the target histogram data distribution of effective feature points under each RGB channel is calculated. Based on the target histogram data distribution of effective feature points under each RGB channel, the target cumulative distribution function of effective feature points under each RGB channel is calculated to obtain the target function set. The initial correction region is also separated into RGB channels, and the initial correction histogram data distribution of effective feature points under each RGB channel is calculated. Based on the initial correction histogram data distribution of effective feature points under each RGB channel, the initial correction cumulative distribution function of effective feature points under each RGB channel is calculated to obtain the initial correction function set.

5. The method for high color accuracy stitching of digital pathological images based on feature point matching according to claim 1, characterized in that, Based on the objective function set and the initial correction function set, histogram normalization is performed on the initial correction region to obtain the first color correction region. With the goal of minimizing the color difference between the first color correction region and the initial correction region, the color mapping function for each RGB channel is calculated to obtain the initial color mapping function. The formula for calculating the color mapping function for each RGB channel is expressed as follows: Wherein, the grayscale value of a k-pixel... This is the mapping function for the pixel grayscale value k in the RGB channel. The coordinates of the valid feature points are numbered. , Let k be the set of valid feature points with pixel color value k in the initial correction region. This is the initial correction region. Valid feature points in the initial correction region pixel color values, This is the first color correction area. Valid feature points in the first color correction region The pixel color value is denoted as c, and the corresponding pixel color value is mapped to c.

6. The method for high-color-accuracy stitching of digital pathological images based on feature point matching according to claim 5, characterized in that, The initial color mapping function is applied to each valid feature point within the initial correction region to obtain the second color correction region. The color difference between the valid feature points in the second color correction region and the corresponding valid feature points in the first color correction region in each RGB channel is calculated. Valid feature points with color differences greater than a set threshold are removed to obtain the updated color correction region. Use the updated version The global color mapping function is obtained by recalculating the color mapping function for each RGB channel.

7. A high-color-accuracy stitching device for digital pathological images based on feature point matching, characterized in that, include: The acquisition module acquires temporally adjacent target splicing images and images to be spliced ​​based on the scanning order of digital pathological images, and acquires the target splicing region in the target splicing image and the region to be spliced ​​in the image to be spliced ​​based on the preset splicing region size. The feature point matching module is used to perform feature point matching between the region to be stitched and the target region to obtain an affine transformation matrix. The affine transformation matrix is ​​then used to perform an affine transformation on the region to be stitched to obtain an initial correction region. Feature points that are correctly matched between the initial correction region and the target region to be stitched are taken as valid feature points. The color correction module is used to calculate the target cumulative distribution function of the effective feature points in each RGB channel within the target stitching area to obtain the target function set, calculate the initial correction cumulative distribution function of the effective feature points in each RGB channel within the initial correction area to obtain the initial correction function set, construct a global color mapping function based on the initial correction function set and the target function set, perform an affine transformation on the image to be stitched using an affine transformation matrix to obtain the initial corrected image, and map the initial corrected image using the global color mapping function to obtain the color corrected image; The stitching module stitches the color-corrected image and the target stitching image according to the scanning order of the pathological images to obtain the digital pathological image stitching result. Specifically, it obtains the overlapping area of ​​the color-corrected image and the target stitching image based on a preset stitching area size, and uses the pixel grayscale difference between two corresponding overlapping pixels in the overlapping area as the pixel energy value of the corresponding pixel. Based on a dynamic programming algorithm, it obtains the optimal stitching seam with the constraint that the stitching seam in the overlapping area is continuous and the sum of the pixel energy values ​​in the stitching seam is minimized. The color-corrected image and the target stitching image are then stitched together based on the optimal stitching seam.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform a high-color-accuracy digital pathological image stitching method based on feature point matching as described in any one of claims 1-6.

9. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements a high-color-accuracy stitching method for digital pathological images based on feature point matching as described in any one of claims 1-6.

Images