Digital Pathology Image Enhancement Method
Through the adaptive dual-template image registration method, the joint evaluation value of probability variance and sliding window sliding are used to solve the problem of large errors and calculations in digital pathological image registration, and higher quality and efficient image fusion are achieved.
Patent Information
- Application Number
- CN202411310420.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-09-20
AI Technical Summary
During the digital pathological image registration process, traditional methods rely on single template registration, which can easily lead to large errors and calculations, and the template selection is not representative enough, affecting image quality.
Adaptive dual-template image registration method is adopted, and the dual-template template is selected through the joint evaluation value of probability variance, combined with sliding window sliding and step size adjustment, to achieve adaptive fusion of the image.
It improves the representativeness and accuracy of template selection, improves image quality and calculation speed, and has higher accuracy and efficiency than single template registration.
Smart Images

Figure CN118822871B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of data processing, and in particular relates to a digital pathological image enhancement method. Background Art
[0002] Digital pathology images are formed by scanning and fusion of pathological tissues after dehydration, embedding, and slicing, and then scanning and fusion by a microscopic scanning device. Compared with the traditional microscope observation of lesion tissues, digital pathology images do not rely on optical microscopes and can be displayed and observed directly on any device, and can magnify a certain part of the slice at a high magnification to observe more subtle tissue morphological changes. Therefore, digital pathology images are of great significance to the development of modern medicine. Digital pathology images are microscopic tissue images of pathological slices scanned by scanning equipment, and then the captured images are fused. There are overlapping parts of adjacent images during the shooting process. The traditional method determines the fusion area through positioning, image registration and other operations. In the registration process, the selection of templates has a great influence on the registration results, and the registration of a single template is prone to cause large errors; at the same time, the registration operation of equal step length using templates in the registration process is computationally intensive. The traditional calculation method of selecting pathological templates does not express the features deeply enough, so the selected templates are not representative enough; and the registration of single template pathological images is inaccurate due to the characteristics of pathological images, and the registration operation of equal step length in the registration process is computationally intensive. Summary of the invention
[0003] The present invention provides a digital pathological image enhancement method, which selects dual templates according to pathological slice images by using a selection function, and performs adaptive dual template image registration according to the pathological slice images, and performs image fusion on the registered images.
[0004] A digital pathology image enhancement method of the present invention adopts the following technical solution: the method comprises:
[0005] Using a scanning device to collect multiple pathological slice images, selecting each of two adjacent pathological slice images as the previous image to be fused and the next image to be fused, and determining the overlapping area of the previous image to be fused and the next image to be fused;
[0006] A sliding window of a fixed size is used to slide in the overlapping area of the previous image to be fused to obtain multiple image blocks in the window, the sliding window of the fixed size is expanded to obtain an expanded sliding window, and the expanded sliding window is used to slide in the overlapping area of the previous image to be fused to obtain multiple expanded image blocks;
[0007] The probability variance joint evaluation value corresponding to each image block is calculated using the grayscale value of each pixel in the multiple image blocks in the windows and the multiple expanded image blocks, and the image blocks corresponding to the first two maximum values of the probability variance joint evaluation value are selected as the first template image block and the second template image block;
[0008] A sliding window of the same size as the first template image block is used to perform a sliding window operation in the overlapping area of the next image to be fused to obtain the current first image block to be registered, and a probability variance joint evaluation value of the current first image block to be registered is calculated; a joint error of the current first image block to be registered is calculated according to the probability variance joint evaluation value of the current first image block to be registered and the first template image block, and a step length of the sliding window to the next position is calculated according to the joint error of the current first image block to be registered; the image block to be registered of the sliding window at the next position is obtained, and similarly, all the image blocks to be registered of the sliding window in the overlapping area of the next image to be fused are obtained, and the joint error value of all the first image blocks to be registered is calculated, and a plurality of first candidate registered image blocks are selected according to the joint error value;
[0009] Similarly, all second candidate registration image blocks in the overlapping area of the next image to be fused with the sliding window of the same size as the second template image block are obtained;
[0010] Calculating the slope of the line connecting each first candidate registration image block and the center point of the first template image block, and the slope of the line connecting each second candidate registration image block and the center point of the second template image block, respectively, and selecting the final first registration image block for matching with the first template image block and the final second registration image block for matching with the second template image block according to all the obtained slopes;
[0011] The first template image block is registered with the final first registered image block, and the second template image block is registered with the final second registered image block to fuse and dock adjacent pathological slice images, and the fusion and docking of all pathological slice images are completed in sequence to obtain the final pathological image.
[0012] Furthermore, the step of expanding the sliding window of a fixed size to obtain an expanded sliding window, and using the expanded sliding window to slide in the overlapping area of the previous image to be fused to obtain multiple expanded image blocks includes:
[0013] Expanding a sliding window of a fixed size multiple times to obtain multiple expanded sliding windows;
[0014] Each expanded sliding window is used to slide in the overlapping area of the previous image to be fused to obtain multiple expanded image blocks corresponding to each expanded sliding window.
[0015] Furthermore, the step of calculating the probability variance joint evaluation value corresponding to each image block by using the grayscale value of each pixel in the multiple image blocks in the windows and the multiple expanded image blocks includes:
[0016] Calculate the probability variance joint evaluation value of the image block in each window according to the gray value of each pixel in the image block in each window;
[0017] After calculating the probability variance joint evaluation value of the image block in each window, sorting them in descending order obtains a probability variance joint evaluation value sequence;
[0018] Selecting a plurality of top-ranked probability variance joint evaluation values from the probability variance joint evaluation value sequence as the screened probability variance joint evaluation values;
[0019] At the same time, multiple image blocks in the window corresponding to each post-screening probability variance joint evaluation value are obtained as multiple post-screening image blocks;
[0020] Selecting a plurality of expanded image blocks expanded based on the plurality of filtered image blocks as filtered expanded image blocks;
[0021] The probability variance joint evaluation value corresponding to each filtered and expanded image block is calculated using the grayscale value of each pixel in the filtered and expanded image block.
[0022] Furthermore, the step of selecting image blocks corresponding to the first two maximum values of the probability variance joint evaluation value as the first template image block and the second template image block includes:
[0023] Select a filtered image block as the target image block;
[0024] Obtaining a probability variance joint evaluation value corresponding to a target image block, and simultaneously obtaining probability variance joint evaluation values corresponding to a plurality of screened expanded image blocks obtained after multiple expansions based on the target image block;
[0025] All the obtained probability variance joint evaluation values are added together as the sum of the probability variance joint evaluation values of the position where the target image block is located;
[0026] According to the calculation method of the sum of the probability variance joint evaluation values at the location of the target image block, the sum of the probability variance joint evaluation values at the location of each filtered image block is calculated, and all the obtained sum values are sorted in descending order to obtain a sum value sequence;
[0027] Select the top two sum values from the sum value sequence, and simultaneously obtain two filtered image blocks corresponding to the top two sum values;
[0028] Two maximum values are selected from the probability variance joint evaluation values corresponding to the two filtered image blocks and the probability variance joint evaluation values corresponding to multiple filtered and expanded image blocks obtained after multiple expansions based on the two filtered image blocks, and the image blocks corresponding to the two maximum values are used as the first template image block and the second template image block.
[0029] Furthermore, the calculation formula for calculating the joint error of the current first image block to be registered based on the probability variance joint evaluation value of the current first image block to be registered and the first template image block is:
[0030]
[0031] in, represents the joint error; represents a probability variance joint evaluation value of the first template image block; Represents the probability variance joint evaluation value of the current image block to be registered; Represents the horizontal coordinate value of the pixel point in the first template image block; Represents the ordinate value of a pixel point in the first template image block; Indicates that the horizontal coordinate of the first template image block is , the vertical axis is The gray value of the pixel; Indicates that the horizontal coordinate of the current image block to be registered is , the vertical axis is The gray value of the pixel.
[0032] Furthermore, the calculation formula for calculating the step length of the sliding window to the next position according to the joint error of the current first image block to be registered is:
[0033]
[0034] in, Indicates the step size of the sliding window sliding to the next position; represents the weight, Represents bias. Both are to avoid step sizes that are too large or too small, and are set based on experience. Represents the joint error of the current image block to be registered.
[0035] Furthermore, the step of calculating the joint error value of all first image blocks to be registered and selecting a plurality of first candidate registered image blocks according to the joint error value comprises:
[0036] After calculating the step length of the sliding window to the next position, the sliding window slides from the current first image block to be registered to the next position in the overlapping area of the next image to be fused according to the step length, and the next position of the sliding window is used as the position of the current first image block to be registered;
[0037] Calculate the joint error value of all first image blocks to be registered during the sliding window sliding process;
[0038] Sorting all obtained joint error values in ascending order to obtain a joint error sequence;
[0039] A plurality of first to-be-registered image blocks with top rankings are selected from the joint error sequence as a plurality of first candidate registration image blocks.
[0040] Furthermore, the step of selecting a final first registered image block for matching with the first template image block and a final second registered image block for matching with the second template image block according to all the obtained slopes includes:
[0041] Calculate the slope of the line connecting the center point of each first candidate registration image block and the first template image block to form a first slope sequence;
[0042] Calculate the slope of the line connecting each second candidate registration image block and the center point of the second template image block to form a second slope sequence;
[0043] Subtract each value in the second slope from each value in the first slope to obtain a slope difference, and select the smallest slope difference from all slope differences;
[0044] Obtaining two slopes corresponding to the minimum slope difference as two post-screening slopes, and simultaneously determining a first candidate registration image block and a second candidate registration image block corresponding to the two post-screening slopes;
[0045] The first candidate registered image block and the second candidate registered image block corresponding to the two filtered slopes are respectively used as the final first registered image block and the final second registered image block.
[0046] The beneficial effects of the present invention are:
[0047] The template selection is optimized by improving the calculation method of registration template selection in the process of adjacent image fusion, and the template selection is adapted according to the characteristics of the pathological image. An adaptive dual-template registration algorithm is proposed, which makes the selected registration template more representative, improves the quality of the fused image, and realizes adaptive enhancement of the pathological image. In the process of template image registration, the registration step size is adaptively changed according to the similarity of each registration, so as to improve the calculation speed. According to the characteristics of dual-template registration, the accuracy is higher than that of single template registration. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0049] Figure 1 A schematic flow chart of the overall steps of an embodiment of a digital pathology image enhancement method of the present invention;
[0050] Figure 2 It is a schematic diagram of fusing and docking a previous image to be fused and a subsequent image to be fused in a digital pathology image enhancement method of the present invention. DETAILED DESCRIPTION
[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0052] An embodiment of a digital pathological image enhancement method of the present invention is as follows: Figure 1 As shown, the method includes:
[0053] S1. Use a scanning device to collect multiple pathological slice images, select each two adjacent pathological slice images as the previous image to be fused and the next image to be fused, and determine the overlapping area of the previous image to be fused and the next image to be fused.
[0054] In the process of scanning the pathological slices, since the moving speed of the scanning device is fixed, in order to comprehensively collect the pathological slice images, the adjacent pathological slice images will have overlapping areas, and the overlapping areas need to be fused when the complete digital pathological slice images are finally formed. Therefore, the overlapping areas need to be determined first. Since the pathological slice images are large images under the microscope mirror, the scanning device needs to scan slowly on the pathological slices, and then fuse the adjacent pathological slice images obtained. Therefore, in order to ensure the clarity of the image, the movement speed of the scanning device is small and uniform, so the overlapping area of each image is roughly fixed, and the overlapping area of each image can be obtained according to the movement speed of the scanning device and the width of the scanning device each time it scans.
[0055] S2, using a sliding window of a fixed size to slide in the overlapping area of the previous image to be fused to obtain multiple image blocks in the window, expanding the sliding window of the fixed size to obtain an expanded sliding window, and using the expanded sliding window to slide in the overlapping area of the previous image to be fused to obtain multiple expanded image blocks.
[0056] Among them, the step of expanding a sliding window of a fixed size to obtain an expanded sliding window, and using the expanded sliding window to slide in the overlapping area of the previous image to be fused to obtain multiple expanded image blocks includes: expanding the sliding window of a fixed size multiple times to obtain multiple expanded sliding windows; using each expanded sliding window to slide in the overlapping area of the previous image to be fused to obtain multiple expanded image blocks corresponding to each expanded sliding window.
[0057] S3. Calculate the probability variance joint evaluation value corresponding to each image block using the grayscale value of each pixel in the multiple image blocks in the windows and the multiple expanded image blocks, and select the image blocks corresponding to the first two largest values of the probability variance joint evaluation value as the first template image block and the second template image block.
[0058] Among them, the step of calculating the probability variance joint evaluation value corresponding to each image block using the grayscale value of each pixel in the multiple image blocks in the windows and the multiple expanded image blocks includes: calculating the probability variance joint evaluation value of each image block in the window according to the grayscale value of each pixel in the image blocks in the window; after calculating the probability variance joint evaluation value of each image block in the window, sorting them in descending order to obtain a probability variance joint evaluation value sequence; selecting a plurality of top-ranked probability variance joint evaluation values from the probability variance joint evaluation value sequence as the post-screening probability variance joint evaluation values; simultaneously obtaining a plurality of image blocks in the windows corresponding to each post-screening probability variance joint evaluation value as a plurality of post-screening image blocks; selecting a plurality of expanded image blocks expanded based on the plurality of filtered image blocks as the filtered expanded image blocks; and calculating the probability variance joint evaluation value corresponding to each filtered expanded image block using the grayscale value of each pixel in the filtered expanded image blocks.
[0059] The step of selecting the image blocks corresponding to the first two maximum values of the probability variance joint evaluation value as the first template image block and the second template image block includes: selecting a screened image block as a target image block; obtaining the probability variance joint evaluation value corresponding to the target image block, and simultaneously obtaining the probability variance joint evaluation values corresponding to a plurality of screened and expanded image blocks obtained after multiple expansions based on the target image block; adding all the obtained probability variance joint evaluation values as the sum of the probability variance joint evaluation values at the location of the target image block; and calculating according to the calculation method of the sum of the probability variance joint evaluation values at the location of the target image block. The sum of the probability variance joint evaluation values at the position of each filtered image block is obtained, and all the obtained sum values are sorted in descending order to obtain a sum value sequence; the first two sum values are selected from the sum value sequence, and the two filtered image blocks corresponding to the first two sum values are obtained at the same time; two maximum values are selected from the probability variance joint evaluation values corresponding to the two filtered image blocks and the probability variance joint evaluation values corresponding to multiple filtered and expanded image blocks obtained after multiple expansions based on the two filtered image blocks, and the filtered image blocks or the filtered and expanded image blocks corresponding to the two maximum values are used as the first template image block and the second template image block.
[0060] At present, in the process of pathological image fusion, all images taken by the microscope need to be fused. Fusion is to fuse the overlapping areas of adjacent images. Therefore, the overlapping areas of two adjacent images need to be registered first. The registration algorithm based on grayscale template is a commonly used registration algorithm in pathological image fusion. This method first determines the reference image block in one image as the template image, and then calculates the image block in another image that matches this image block for registration.
[0061] The traditional template is determined by finding the image block T with the most information in the image as the grayscale template based on the size of the variance, and the size of the grayscale template T is fixed. Then, based on the information of T, a sliding window operation is performed in the overlapping area of another image to find the image block most similar to T, and then use this as a reference for image registration. The above method is prone to errors in firstly obtaining the measurement of information amount based only on the variance, and the fixed-size image block T is prone to contrast errors in the sliding window process. For example, the sliding window results of multiple positions are very similar, making it difficult to distinguish. At the same time, since the pathological images basically show the frontal images of cells, the images between many regions are very similar, so the method of determining the grayscale template image by only using the variance to measure the amount of information contained is prone to misjudgment.
[0062] The present invention determines the amount of information contained in the current pathological image block by constructing a probability variance joint evaluation function. The amount of information is the probability variance joint evaluation value corresponding to each image block.
[0063]
[0064] in, Represents the probability variance joint evaluation value corresponding to the current image block; Indicates the number of different gray values that appear; Indicates the number of The probability of the gray value corresponding to the pixel; Indicates The gray value of each pixel; Represents the average gray value of the current image block. For the above formula, the probability of gray value and the number of gray values are introduced into the original variance calculation formula. This method can make the small differences between different images have larger differences in the final quantization, thereby making it easier to calculate the gray template image.
[0065] The present invention improves the quantization method for determining the grayscale template image during image fusion in the prior art. The traditional algorithm determines the size of T and then calculates it in the overlapping images, and selects the maximum variance value as the template image. However, for pathological images, due to the similarity of many areas, this fixed size method is prone to errors. It is possible that under the current size of T, the variance of the image block is the largest, but if it is slightly enlarged or reduced, it will change. Therefore, the present invention realizes the selection of the optimal template image by adaptively adapting the size of the grayscale template image.
[0066] First, a fixed-size window T is used to perform a sliding window scan on the overlapping area of adjacent images to be fused. Each time the window slides, an image block in the window is obtained. The probability variance joint evaluation value of each image block in the window is calculated, and the probability variance joint evaluation value of different positions is also obtained:
[0067]
[0068] The above formula is the probability variance joint evaluation value of different positions in the overlapping area. The traditional approach is to select the maximum value as the grayscale template image, but the pathological template image determined in this way is prone to errors. It can be known that the possibility of errors in general will only appear between the largest values, such as misjudgment between the maximum value and the two values next to the maximum value. Therefore, after obtaining a set of probability variance joint evaluation values, the present invention selects the top five data in numerical ranking and conducts in-depth analysis to determine the optimal pathological image template.
[0069] When determining the pathological template image, the larger the template image area, the higher the accuracy of the final pathological image registration, because the larger the image area, the richer the feature information it contains. Therefore, when correcting the template image, the image blocks in the five windows corresponding to the five data with the top five numerical values are used as the five screened image blocks.
[0070] Take the five filtered image blocks as candidate boxes, expand the range and recalculate with them as the center.
[0071]
[0072] in, Represents the probability variance joint evaluation value corresponding to the first filtered image block; represents the probability variance joint evaluation value corresponding to the second filtered image block; represents the set of probability variance joint evaluation values corresponding to the five filtered image blocks when they are not expanded;
[0073] The expansion method is to expand the length, width and height of the fixed-size sliding window by (1 / 7, 1 / 5, 1 / 3) respectively, and then obtain the sliding window expanded by 1 / 7, the sliding window expanded by 1 / 5 and the sliding window expanded by 1 / 3. The sliding window expanded by 1 / 7, the sliding window expanded by 1 / 5 and the sliding window expanded by 1 / 3 are used to slide in the overlapping area of the previous image to be fused to obtain multiple groups of expanded image blocks, each group of expanded image blocks contains multiple expanded image blocks.
[0074] Calculate the probability joint variance value of each expanded image block in each group of expanded image blocks. A total of three sets of probability joint variance sets are obtained:
[0075]
[0076]
[0077]
[0078] In the above formula, BCD represents the set calculated after expanding (1 / 7, 1 / 5, 1 / 3) three times. Indicates The filtered image block The corresponding probability variance joint evaluation value after the expansion.
[0079] The present invention selects 5 filtered image blocks to calculate the probability variance joint evaluation values corresponding to the 5 filtered image blocks after 3 expansions in order to reduce the amount of calculation. The present invention adds the four probability variance joint evaluation values obtained after 3 expansions of a filtered image block as the sum of the probability variance joint evaluation values at the location of the filtered image block. The calculation formula for the sum of the probability variance joint evaluation values at the same location is:
[0080]
[0081]
[0082] Fixed-size template images may have errors, so we calculate the sum of the joint variances of the expanded probabilities based on the fixed-size templates, and select the images after joint expansion. The two largest regions are used as candidate regions for the pathological template image.
[0083]
[0084]
[0085] In the above formula, the two largest values are used to determine the two filtered image blocks with the largest sum of probability variance joint evaluation values among the five filtered image blocks. Then, The maximum value is selected from the corresponding unexpanded filtered image block and the three-expanded image block as the corresponding image block as the first template image block and the second template image block.
[0086]
[0087]
[0088] Then the final first template image block and the second template image block are The corresponding template size is recorded as .
[0089] S4, using a sliding window of the same size as the first template image block to perform a sliding window operation in the overlapping area of the next image to be fused, obtain the current first image block to be registered, and calculate the probability variance joint evaluation value of the current first image block to be registered; calculate the joint error of the current first image block to be registered according to the probability variance joint evaluation value of the current first image block to be registered and the first template image block, and calculate the step size of the sliding window to the next position according to the joint error of the current first image block to be registered; obtain the image block to be registered of the sliding window at the next position, and similarly obtain all the image blocks to be registered of the sliding window in the overlapping area of the next image to be fused, and calculate the joint error value of all the first image blocks to be registered, and select multiple first candidate registered image blocks according to the joint error value.
[0090] The present invention determines the final first template image block and the second template image block in step S3. In this step, it is necessary to perform image registration in the overlapping area of another image according to the template image of the pathology. The so-called image registration is to find the best position for image fusion, because although the overlapping area has been determined, it is necessary to fuse the overlapping areas of the two adjacent images, and the fusion needs to determine the specific direction, position, and other information of the fusion, so it is necessary to determine the specific information of the fusion according to the template image. The traditional method is to perform a sliding window operation in another image according to the template pathology image, calculate the similarity between the image under each window and the template image, and the step size of the sliding window is fixed. The first disadvantage of the above method is that the amount of calculation is large, because the window determined by the template is fixed, so there is only one window in the image to be registered that is correct, so the fixed step size will lead to a significant increase in the calculation, and because most areas of the pathology image are similar, they are all cell images after staining, so there is only one template image, and it is easy to have a registration error when re-registering, and it cannot accurately locate the fusion area, which leads to the poor final imaging effect of the digital pathology image. The present invention realizes adaptive enhancement of pathological images by performing a dual-template adaptive step-size registration operation on two determined template images.
[0091] When using the pathological template image for registration, ideally the probability joint variance value of the image to be registered should be consistent with that of the pathological template image. Therefore, the mean square error function is combined to improve the accuracy of the registration process. A sliding window of the same size as the first template image block is used to perform a sliding window operation in the overlapping area of the next image to be fused to obtain the current first image block to be registered, and the probability variance joint evaluation value of the current image block to be registered is calculated; the calculation method of the probability variance joint evaluation value of the current image block to be registered is the same as the calculation method of the probability variance joint evaluation value corresponding to the image block in the window, and both are calculated using the grayscale value of each pixel in the current image block to be registered.
[0092] After calculating the probability variance joint evaluation value of the current image block to be registered, the joint error of the current image block to be registered is calculated according to the probability variance joint evaluation value of the current image block to be registered and the first template image block. The calculation formula of the joint error is:
[0093]
[0094] in, represents the joint error; represents a probability variance joint evaluation value of the first template image block; Represents the probability variance joint evaluation value of the current image block to be registered; Represents the horizontal coordinate value of the pixel point in the first template image block; Represents the ordinate value of a pixel point in the first template image block; Indicates that the horizontal coordinate of the first template image block is , the vertical axis is The gray value of the pixel; Indicates that the horizontal coordinate of the current image block to be registered is , the vertical axis is The grayscale value of the pixel point. Based on the mean square error, the probability variance joint evaluation value is introduced to measure the difference between the sliding windows in the pathological template image and the image to be registered. The smaller the difference, the greater the quantitative difference between the optimal window and the remaining windows in the image to be registered, which makes the selection of the ideal window more accurate.
[0095] At the same time, the present invention can also adjust the step size of the sliding window according to the joint error between the image block to be registered and the template image block. The larger the value, the greater the difference between the image block to be registered and the template image block. Due to the correlation between images, the difference between the image block to be registered and the template image block around the sliding window should also be large. Therefore, the step size of the sliding window should be expanded at this time to reduce the calculation; on the contrary, if the difference is small, it indicates that there is a high similarity between the current image block to be registered and the template image block. Combined with the correlation characteristics between images, the image block to be registered around the window and the template image block are also similar. In order to more accurately determine the final registered image, the step size should be reduced.
[0096] The formula for calculating the step length of the sliding window to the next position based on the joint error of the current image block to be registered is:
[0097]
[0098] in, Indicates the step size of the sliding window sliding to the next position; represents the weight, Represents bias. Both are to avoid step sizes that are too large or too small, and are set based on experience. Represents the joint error of the current image block to be registered. The smaller the joint error, the smaller the step size should be, and the larger the joint error, the larger the step size should be. However, the relationship between the joint error and the step size is not a simple linear relationship, but a nonlinear one. This can better save calculations and reduce the number of sliding windows.
[0099] Among them, the joint error value of all the first image blocks to be registered is calculated in the same way, and the step of selecting multiple first candidate image blocks to be registered according to the joint error value includes: after calculating the step length of the sliding window to slide to the next position, the sliding window slides from the current first image block to be registered to the next position in the overlapping area of the next image to be fused according to the step length, and the next position of the sliding window is used as the position of the current first image block to be registered; in the process of sliding the sliding window, the joint error value of all the first image blocks to be registered is calculated; all the obtained joint error values are sorted in ascending order to obtain a joint error sequence; and multiple first image blocks to be registered that are ranked high in the joint error sequence are selected as multiple first candidate image blocks to be registered.
[0100] S5. Similarly, all second candidate registration image blocks in the overlapping area of the next image to be fused are obtained in the sliding window of the same size as the second template image block.
[0101] A sliding window of the same size as the second template image block is used to perform a sliding window operation in the overlapping area of the subsequent image to be fused to obtain the current second image block to be registered, and a probability variance joint evaluation value of the current second image block to be registered is calculated; a joint error of the current second image block to be registered is calculated based on the probability variance joint evaluation value of the current second image block to be registered and the second template image block, and a step size of sliding the sliding window to the next position is calculated based on the joint error of the current second image block to be registered; similarly, the joint error value of all second image blocks to be registered is calculated, and multiple second candidate registration image blocks are selected based on the joint error value.
[0102] In the process of sliding window image registration, the traditional method is to perform image registration operation of a single template image with a fixed step size. The present invention determines the two optimal first template image blocks and the second template image blocks and then performs image registration of dual templates with adaptive step sizes. Compared with the registration of a single template with a fixed step size, the registration accuracy of the present invention is higher and the quality of the obtained fused pathological image is higher.
[0103] S6. Calculate the slope of the line connecting each first candidate registration image block and the center point of the first template image block, and the slope of the line connecting each second candidate registration image block and the center point of the second template image block, and select the final first registration image block for matching with the first template image block and the final second registration image block for matching with the second template image block according to all the obtained slopes.
[0104] Among them, the step of selecting the final first registered image block for matching with the first template image block and the final second registered image block for matching with the second template image block according to all the slopes obtained includes: calculating the slope of the line connecting each first candidate registered image block and the center point of the first template image block to form a first slope sequence; calculating the slope of the line connecting each second candidate registered image block and the center point of the second template image block to form a second slope sequence; subtracting each value in the second slope from each value in the first slope to obtain a slope difference, and selecting the smallest slope difference from all the slope differences; obtaining two slopes corresponding to the smallest slope difference as two post-screening slopes, and simultaneously determining the first candidate registered image block and the second candidate registered image block corresponding to the two post-screening slopes; and using the first candidate registered image block and the second candidate registered image block corresponding to the two post-screening slopes as the final first registered image block and the final second registered image block, respectively.
[0105] In the present invention, each value in the second slope is subtracted from each value in the first slope to obtain a slope difference. If 5 first candidate registration image blocks are selected, there will be 5 values in the first slope sequence. At the same time, if 5 second candidate registration image blocks are selected, there will be 5 values in the second slope sequence. Therefore, each value in the second slope is subtracted from each value in the first slope to obtain a slope difference. A total of 25 slope differences need to be calculated. However, as a fusion process of pathological images, the accuracy of pathological images is the most important requirement, so the increase in the amount of calculation brought about is completely acceptable.
[0106] S7, using the first template image block to register with the final first registered image block, and the second template image block to register with the final second registered image block to fuse and dock adjacent pathological slice images, and completing the fusion and docking of all pathological slice images in sequence to obtain a final pathological image.
[0107] In the present invention, 25 slope differences are obtained in step S6, and the minimum value is selected from the 25 slope differences. . , at this time, the two template registration windows should be fused to obtain the fused image. In the actual calculation process, there may be errors caused by the calculation. However, under normal circumstances The value of is also very small. At this time, in order to ensure the accuracy of the registration, the final first registered image block or the final second registered image block corresponding to the minimum value of the joint error should be selected from the final first registered image block and the final second registered image block as the best registered image block, and the remaining other registered image block should be used as the reference registered image block.
[0108] The first template image block is registered with the final first registered image block, and the second template image block is registered with the final second registered image block to fuse and dock the adjacent pathological slice images, such as Figure 2 , which is a schematic diagram of fusing the previous image to be fused and the next image to be fused. The fusion and docking of all pathological slice images are completed in sequence to obtain the final pathological image.
[0109] In summary, the present invention provides a digital pathological image enhancement method and proposes an adaptive dual-template registration algorithm, so that the selected registration template is more representative, the fused image quality is higher, and the adaptive enhancement of the pathological image is achieved; and in the process of using the template image registration, the registration step size is adaptively changed according to the similarity of each registration, so as to improve the calculation speed, and according to the characteristics of the dual-template registration, the accuracy is higher than that of the single template registration.
Claims
1. A digital pathological image enhancement method, characterized in that: The method includes: Using a scanning device to collect multiple pathological slice images, selecting each of two adjacent pathological slice images as the previous image to be fused and the next image to be fused, and determining the overlapping area of the previous image to be fused and the next image to be fused; A sliding window of a fixed size is used to slide in the overlapping area of the previous image to be fused to obtain multiple image blocks in the window, the sliding window of the fixed size is expanded to obtain an expanded sliding window, and the expanded sliding window is used to slide in the overlapping area of the previous image to be fused to obtain multiple expanded image blocks; The probability variance joint evaluation value corresponding to each image block is calculated using the grayscale value of each pixel in the multiple image blocks in the windows and the multiple expanded image blocks, and the image blocks corresponding to the first two maximum values of the probability variance joint evaluation value are selected as the first template image block and the second template image block; A sliding window of the same size as the first template image block is used to perform a sliding window operation in the overlapping area of the next image to be fused to obtain the current first image block to be registered, and a probability variance joint evaluation value of the current first image block to be registered is calculated; a joint error of the current first image block to be registered is calculated according to the probability variance joint evaluation value of the current first image block to be registered and the first template image block, and a step length of the sliding window to the next position is calculated according to the joint error of the current first image block to be registered; the image block to be registered of the sliding window at the next position is obtained, and similarly, all the image blocks to be registered of the sliding window in the overlapping area of the next image to be fused are obtained, and the joint error value of all the first image blocks to be registered is calculated, and a plurality of first candidate registered image blocks are selected according to the joint error value; Similarly, all second candidate registration image blocks in the overlapping area of the next image to be fused with the sliding window of the same size as the second template image block are obtained; Calculating the slope of the line connecting each first candidate registration image block and the center point of the first template image block, and the slope of the line connecting each second candidate registration image block and the center point of the second template image block, respectively, and selecting the final first registration image block for matching with the first template image block and the final second registration image block for matching with the second template image block according to all the obtained slopes; The first template image block is registered with the final first registered image block, and the second template image block is registered with the final second registered image block to fuse and dock adjacent pathological slice images, and the fusion and docking of all pathological slice images are completed in sequence to obtain the final pathological image.
2. A digital pathological image enhancement method according to claim 1, characterized in that: The step of expanding the sliding window of fixed size to obtain an expanded sliding window, and using the expanded sliding window to slide in the overlapping area of the previous image to be fused to obtain multiple expanded image blocks includes: Expanding a sliding window of a fixed size multiple times to obtain multiple expanded sliding windows; Each expanded sliding window is used to slide in the overlapping area of the previous image to be fused to obtain multiple expanded image blocks corresponding to each expanded sliding window.
3. A digital pathological image enhancement method according to claim 2, characterized in that: The step of calculating the probability variance joint evaluation value corresponding to each image block by using the grayscale value of each pixel in the multiple image blocks in the windows and the multiple expanded image blocks comprises: Calculate the probability variance joint evaluation value of the image block in each window according to the gray value of each pixel in the image block in each window; After calculating the probability variance joint evaluation value of the image block in each window, sorting them in descending order obtains a probability variance joint evaluation value sequence; Selecting a plurality of top-ranked probability variance joint evaluation values from the probability variance joint evaluation value sequence as the screened probability variance joint evaluation values; At the same time, multiple image blocks in the window corresponding to each post-screening probability variance joint evaluation value are obtained as multiple post-screening image blocks; Selecting a plurality of expanded image blocks expanded based on the plurality of filtered image blocks as filtered expanded image blocks; The probability variance joint evaluation value corresponding to each filtered and expanded image block is calculated using the grayscale value of each pixel in the filtered and expanded image block.
4. A digital pathological image enhancement method according to claim 3, characterized in that: The step of selecting the image blocks corresponding to the first two maximum values of the probability variance joint evaluation value as the first template image block and the second template image block comprises: Select a filtered image block as the target image block; Obtaining a probability variance joint evaluation value corresponding to a target image block, and simultaneously obtaining probability variance joint evaluation values corresponding to a plurality of screened expanded image blocks obtained after multiple expansions based on the target image block; Adding all the obtained probability variance joint evaluation values as the sum of the probability variance joint evaluation values of the position where the target image block is located; According to the calculation method of the sum of the probability variance joint evaluation values at the location of the target image block, the sum of the probability variance joint evaluation values at the location of each filtered image block is calculated, and all the obtained sum values are sorted in descending order to obtain a sum value sequence; Select the top two sum values from the sum value sequence, and simultaneously obtain two filtered image blocks corresponding to the top two sum values; Two maximum values are selected from the probability variance joint evaluation values corresponding to the two filtered image blocks and the probability variance joint evaluation values corresponding to multiple filtered and expanded image blocks obtained after multiple expansions based on the two filtered image blocks, and the image blocks corresponding to the two maximum values are used as the first template image block and the second template image block.
5. A digital pathological image enhancement method according to claim 1, characterized in that: The calculation formula for calculating the joint error of the current first image block to be registered based on the probability variance joint evaluation value of the current first image block to be registered and the first template image block is: in, represents the joint error; represents a probability variance joint evaluation value of the first template image block; Represents the probability variance joint evaluation value of the current image block to be registered; Represents the horizontal coordinate value of the pixel point in the first template image block; Represents the ordinate value of a pixel point in the first template image block; Indicates that the horizontal coordinate of the first template image block is , the vertical axis is The gray value of the pixel; Indicates that the horizontal coordinate of the current image block to be registered is , the vertical axis is The gray value of the pixel.
6. A digital pathological image enhancement method according to claim 5, characterized in that: The calculation formula for calculating the step length of the sliding window to the next position according to the joint error of the current first image block to be registered is: in, Indicates the step size of the sliding window sliding to the next position; represents the weight, Represents bias. Both are to avoid step sizes that are too large or too small, and are set based on experience. Represents the joint error of the current image block to be registered.
7. A digital pathological image enhancement method according to claim 1, characterized in that: The step of calculating the joint error value of all first image blocks to be registered and selecting a plurality of first candidate registered image blocks according to the joint error value comprises: After calculating the step length of the sliding window to the next position, the sliding window slides from the current first image block to be registered to the next position in the overlapping area of the next image to be fused according to the step length, and the next position of the sliding window is used as the position of the current first image block to be registered; Calculate the joint error value of all first image blocks to be registered during the sliding window sliding process; Sorting all obtained joint error values in ascending order to obtain a joint error sequence; A plurality of first to-be-registered image blocks with top rankings are selected from the joint error sequence as a plurality of first candidate registration image blocks.
8. A digital pathological image enhancement method according to claim 1, characterized in that: The step of selecting a final first registered image block for matching with the first template image block and a final second registered image block for matching with the second template image block according to all the obtained slopes comprises: Calculate the slope of the line connecting the center point of each first candidate registration image block and the first template image block to form a first slope sequence; Calculate the slope of the line connecting each second candidate registration image block and the center point of the second template image block to form a second slope sequence; Subtract each value in the second slope from each value in the first slope to obtain a slope difference, and select the smallest slope difference from all slope differences; Obtaining two slopes corresponding to the minimum slope difference as two post-screening slopes, and simultaneously determining a first candidate registration image block and a second candidate registration image block corresponding to the two post-screening slopes; The first candidate registered image block and the second candidate registered image block corresponding to the two filtered slopes are respectively used as the final first registered image block and the final second registered image block.
