Real-time illumination correction method and device for optical microscopic full-dose image and storage medium

Abstract

The invention relates to a real-time illumination correction method and device for an optical microscopic total image and a storage medium, and the method comprises the steps: S1, introducing basic information, and employing a blank slide to shoot and collect a BGR color image; s2, introducing a reference value, and calculating an average value of pixel values of different color channels of the BGR color image acquired based on blank slide shooting; s3, collecting a plurality of images of the total image of the sample slide; and S4, performing illumination correction on each image of the full-amount image of the sample slide in the step S3 in real time, performing illumination correction on each image of the full-amount image of the sample slide in the step S3 in real time through the basic information introduced in the step S1 and the reference value introduced in the step S2, and finally obtaining a panorama of the full-amount image of the sample slide after illumination correction. The method is efficient and stable, can be applied to different devices, and does not affect the illumination correction effect along with the attenuation of a light path system, the wear of hardware and other reasons; and real-time operation can be well carried out even on a low-computing-power chip.

Description

Technical Field

[0001] The present invention relates to the technical field of uniform illumination and correction, and particularly relates to a real-time illumination correction method, device and storage medium for optical microscopic full-scale images. Background Art

[0002] An automated optical scanning microscope can automatically scan and capture pictures of all fields of an entire sample slide through mechanical automation control, forming a full-scale image of the entire slide. Since there is a problem of uneven illumination in each picture, when observing the full-scale image, the problem of uneven illumination is more likely to be prominent. Therefore, the illumination correction of the full-scale image has become a technical problem urgently to be solved in this field.

[0003] With the rapid development of artificial intelligence, the Chinese invention patent "A Method for Illumination Correction of Medical Microscopic Images Based on Deep Learning" with the publication number CN113570504A and the paper "Correction of uneven illumination in color microscopic image based on fully convolutional network" published in 2021 proposed to use artificial intelligence technology to deal with the problem of full-scale image illumination correction. However, in practical applications, the degree and position of uneven illumination of each microscope are different. It is necessary to make a training set for each device and train a corresponding artificial intelligence model for illumination correction for each device. The workload is very large and cumbersome and repetitive. Moreover, the running speed of the artificial intelligence model during deployment is affected by the computing power of the chip, and a high-cost computing power graphics card is required, and the real-time performance is poor when running on an ordinary CPU. In addition, the above methods also have the following problems. They all belong to the post-processing methods of first collecting all the full-scale images of the sample slide and then processing them, resulting in a long time and low efficiency to obtain a full-scale image with uniform illumination. Therefore, it is particularly important to develop a high-efficiency, stable, and applicable to different devices, and can perform real-time operation well even on low-computing-power chips, full-scale image illumination correction method. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the above-mentioned defects of the prior art and provide a real-time illumination correction method, device and storage medium for optical microscopic full-scale images. This method is efficient, stable, applicable to different devices, and will not affect the illumination correction effect due to reasons such as the attenuation of the optical path system and the wear of the hardware; it can perform real-time operation well even on low-computing-power chips.

[0005] The present invention is achieved by the following technical solutions:

[0006] According to the first aspect of the present application, a real-time illumination correction method for optical microscopic full-scale images is provided, including the following steps:

[0007] S1. Introduce basic information and capture a BGR color image using a blank glass slide.

[0008] S2. Introduce a reference value and calculate the average value of the pixel values of different color channels based on the BGR color image in S1.

[0009] S3. Capture several images of the full image of the sample slide.

[0010] S4. Perform real-time illumination correction on each image of the full image of the sample slide in S3. Perform real-time illumination correction on each image of the full image of the sample slide in S3 through the basic information introduced in S1 and the reference value introduced in S2. The steps are as follows:

[0011] S41. Divide the pixel value of the sample image in S3 by the pixel value of the blank glass slide image at the corresponding position in S1 and multiply by the average value of the pixel values of the corresponding channel of the blank glass slide image in S2.

[0012] S42. Convert the decimal number obtained in S41 into an integer.

[0013] S43. Perform truncation processing on the value obtained in S42 and convert it into a pixel value.

[0014] S44. Obtain the corresponding result after illumination correction. Combine the images of the full image of the sample slide after illumination correction in S43 according to the shooting order and position to obtain a panoramic view of the full image of the sample slide after illumination correction.

[0015] In another embodiment, when capturing a BGR color image using a blank glass slide in S1, the specific steps are as follows: Place a blank and clean transparent glass slide under the objective lens of an optical microscope, and capture a BGR color image under the microscope through a connected camera, denoted as Blank, with a resolution of H×W, that is, the height is H and the width is W, Blank uchar (h, w, c) represents the pixel value of the c-th channel, the h-th row, and the w-th column of the image Blank. c = 0 represents the blue B channel, c = 1 represents the green G channel, and c = 2 represents the red R channel. This value is of uchar type in computer storage, that is, the value range of this value is 0 ≤ Blank uchar (h, w, c) ≤ 255, and it is an integer, where 0 ≤ h ≤ H - 1, 0 ≤ w ≤ W - 1, 0 ≤ c ≤ 2, and h, w, c are all integers.

[0016] In another embodiment, when calculating the average value of the pixel values of different color channels based on the BGR color image captured by the blank glass slide in S2, the calculation formula is:

[0017]

[0018] Among them, represents the average value of all pixel values of the 0th channel, i.e., the blue B channel, represents the average value of all pixel values of the 1st channel, i.e., the green G channel, represents the average value of all pixel values of the 2nd channel, i.e., the red R channel, and for the sake of precision, the calculation result is reserved as a decimal, and the computer stores this value as a float type.

[0019] In another embodiment, in S4, real-time illumination correction is performed on each image of the full-scale image of the sample slide. The specific steps are as follows: Assume that the full-scale image collected contains a total of N images, and the nth image is denoted as I (n) , and the resolution is the same as that of the image Blank. represents the pixel value of the cth channel, the hth row, and the wth column of the nth image I (n) , and assume that I (n) is denoted as O after illumination correction (n) , and the specific steps of illumination correction are as follows:

[0020] S41. Construct a three-dimensional array, denoted as This three-dimensional array has H rows, W columns, and 3 channels. The value of the cth channel, the hth row, and the wth column is calculated by the formula:

[0021]

[0022] Among them, is the pixel value of the cth channel, the hth row, and the wth column of the nth sample image, and Blank uchar (h, w, c) is the pixel value of the cth channel, the hth row, and the wth column of the nth blank slide image, is the average value of all pixel values of the cth channel of the blank slide image. The calculation result is reserved as a decimal, and the computer stores this value as a float type, where 0 ≤ h ≤ H - 1, 0 ≤ w ≤ W - 1, 0 ≤ c ≤ 2, 1 ≤ n ≤ N, and h, w, c, n are all integers;

[0023] S42. Construct a three-dimensional array, denoted as This three-dimensional array has H rows, W columns, and 3 channels. The value of the cth channel, the hth row, and the wth column is calculated by the formula:

[0024]

[0025] Among them, the function round() represents rounding a numerical value to an integer, that is, the value of The value rounded to an integer is stored by the computer as an int type, where 0 ≤ h ≤ H - 1, 0 ≤ w ≤ H - 1, 0 ≤ c ≤ 2, 1 ≤ n ≤ N, and h, w, c, and n are all integers;

[0026] S43. Truncate the value in S42 with a reference of 255 to convert it into a pixel value. The calculation formula for truncation is:

[0027]

[0028] where, is the pixel value of the c-th channel, h-th row, and w-th column of the image O after illumination correction (n) ;

[0029] S44. Obtain the corresponding result after illumination correction. Combine the images of the full-scale sample slide after illumination correction in S43 according to the shooting order and position to finally obtain the panoramic image of the full-scale sample slide after illumination correction.

[0030] In another embodiment, the three-dimensional array constructed in S41 is denoted as This three-dimensional array has H rows, W columns, and 3 channels. The value of the c-th channel, h-th row, and w-th column is calculated by the formula:

[0031]

[0032] where, is the pixel value of the c-th channel, h-th row, and w-th column of the n-th sample image, and Blank uchar (h, w, c) is the pixel value of the c-th channel, h-th row, and w-th column of the n-th blank slide image, is the average value of all pixel values of the c-th channel of the blank slide image, α is the brightness adjustment coefficient, and the calculation result is retained as a decimal. The computer stores this value as a float type, where 0 ≤ h ≤ H - 1, 0 ≤ w ≤ W - 1, 0 ≤ c ≤ 2, 1 ≤ n ≤ N, α > 0, and h, w, c, and n are all integers.

[0033] In another embodiment, the resolution is H × W = 1200 × 1920, that is, the height H = 1200 and the width W = 1920.

[0034] In another embodiment, α = 1.

[0035] In another embodiment, α = 1.5.

[0036] According to the second aspect of the present application, a device is provided, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method in the above first aspect are implemented.

[0037] According to the third aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method in the first aspect are implemented.

[0038] The beneficial effects of the present invention are:

[0039] A real-time illumination correction method, device and storage medium for an optical microscopic full-scale image provided by the present invention, the method includes: S1. Introduce basic information, and use a blank glass slide to capture a BGR color image; S2. Introduce a reference value, and calculate the average value of the pixel values of different color channels of the BGR color image captured by using the blank glass slide; S3. Capture several images of the full-scale image of the sample slide; S4. Perform real-time illumination correction on each image of the full-scale image of the sample slide in S3, and perform real-time illumination correction on each image of the full-scale image of the sample slide in S3 through the basic information introduced in S1 and the reference value introduced in S2, and finally obtain a panoramic view of the full-scale image of the sample slide after illumination correction. The present invention is efficient, stable, and can be applied to different devices, and will not affect the effect of illumination correction due to reasons such as attenuation of the optical path system and wear of the hardware; even on a low-computing power chip, it can perform real-time operations well. Description of the Drawings

[0040] The following further describes a real-time illumination correction method, device and storage medium for an optical microscopic full-scale image with reference to the drawings:

[0041] Figure 1 is a schematic flowchart of a real-time illumination correction method for an optical microscopic full-scale image provided by an embodiment of the present application;

[0042] Figure 2 is a schematic diagram of a BGR color image captured by using a blank glass slide provided by an embodiment of the present application;

[0043] Figure 3 is a schematic diagram of the original full-scale image of a sample slide containing 84 images captured by an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of viewing a panoramic view of the original full-scale image of a sample slide according to the shooting order and position provided by an embodiment of the present application;

[0045] Figure 5 is a schematic diagram of a panoramic view of the full-scale image after illumination correction when the brightness coefficient is low provided by an embodiment of the present application;

[0046] Figure 6 is a schematic diagram of a panoramic view of the full-scale image after illumination correction when the brightness coefficient is high provided by an embodiment of the present application. Specific Embodiments

[0047] The following will further describe the present application in detail in conjunction with the accompanying drawings and embodiments. It should be specifically noted that the following embodiments are only used to illustrate the present application, but do not limit the scope of the present application. Similarly, the following embodiments are only some embodiments of the present application rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0048] Please refer to Figure 1 , a real-time illumination correction method for an optical microscopic full-scale image, including the following steps:

[0049] S1. Introduce basic information, and use a blank glass slide to capture a BGR color image. Place the blank and clean transparent glass slide under the objective lens of an optical microscope, and capture a BGR color image under the microscope through a connected camera. As Figure 2 shown, assume it is denoted as Blank, with a resolution of H×W = 1200×1920, that is, the height H = 1200 and the width W = 1920. Blank uchar (h, w, c) represents the pixel value of the c-th channel, the h-th row, and the w-th column of the image Blank. c = 0 represents the blue B channel, c = 1 represents the green G channel, and c = 2 represents the red R channel. This value is of the uchar type in computer storage, that is, the value range of this value is 0 ≤ Blank uchar (h, w, c) ≤ 255, and it is an integer, where 0 ≤ h ≤ 1199, 0 ≤ w ≤ 1919, 0 ≤ c ≤ 2, and h, w, c are all integers.

[0050] S2. Introduce a reference value, and calculate the average value of the pixel values of the three color channels of the BGR color image captured based on the blank slide. represents the average value of all pixel values of the 0-th channel, that is, the blue B channel. And for the sake of accuracy, the calculation result is retained with decimals. The computer stores this value as a float type. The calculation formula is:

[0051]

[0052] represents the average value of all pixel values of the 1-st channel, that is, the green G channel. And for the sake of accuracy, the calculation result is retained with decimals. The computer stores this value as a float type. The calculation formula is:

[0053]

[0054] Represents the average value of all pixel values of the second channel, i.e., the red R channel. For the sake of precision, the calculation result is retained as a decimal, and the computer stores this value as a float type. The calculation formula is:

[0055]

[0056] S3. Collect several images of the full-scale image of the sample slide.

[0057] S4. Perform illumination correction on the full-scale image collected from the slide with the sample in real time. Assume that the full-scale image collected contains a total of N images. In this embodiment, N = 84, as Figure 3 shown. To more prominently highlight the problem of uneven illumination, view the panoramic image of the original full-scale image according to the shooting order and position. As Figure 4 shown, it can be clearly seen that there is a problem of uneven illumination. Some positions are darker and some are brighter, resulting in streaks when viewing the panoramic image, which affects the observation of relevant personnel. The nth image is denoted as I( n ), and its resolution is the same as that of the image Blank, with a resolution of H × W = 1200 × 1920, that is, the height H = 1200 and the width W = 1920. Represents the pixel value of the cth channel, hth row, and wth column of the nth image I (n) . Assume that after the illumination correction of I (n) it is denoted as O (n) . The specific steps of the illumination correction are as follows:

[0058] S41. Construct a three-dimensional array denoted as This three-dimensional array has H rows, W columns, and 3 channels. The value of the cth channel, hth row, and wth column is The calculation formula is:

[0059]

[0060] Among them, is the pixel value of the cth channel, hth row, and wth column of the nth sample image, and Blank uchar (h, w, c) is the pixel value of the cth channel, hth row, and wth column of the nth blank slide image. is the average value of all pixel values of the cth channel of the blank slide image. Although the numerator and denominator are integers, for the sake of precision, the calculation result after division is retained as a decimal. is a decimal, α is the brightness adjustment coefficient, and it is also a decimal. Then the result after multiplying the three is is also a decimal number. The calculation result is retained as a decimal number, and the computer stores this value as a float type, where 0 ≤ h ≤ H - 1, 0 ≤ w ≤ W - 1, 0 ≤ c ≤ 2, 1 ≤ n ≤ N, α > 0, and h, w, c, and n are all integers.

[0061] S42. Construct a three-dimensional array two and denote it as This three-dimensional array two has H rows, W columns, and 3 channels. The value at the h-th row, w-th column, and c-th channel of the three-dimensional array two is calculated by the formula:

[0062]

[0063] Among them, the function round() represents rounding a numerical value to an integer, that is The value of is the value obtained by rounding the decimal

[0064] S43. Assign a value to the pixel value at the h-th row, w-th column, and c-th channel of the image O (n) after illumination correction. The assignment method is to truncate the value in S42 (n) with 255 as the reference and convert it into a pixel value. The calculation formula for truncation processing is:

[0065]

[0066] Among them, is the pixel value at the h-th row, w-th column, and c-th channel of the image O (n) after illumination correction.

[0067] S44. Obtain the corresponding result after illumination correction, and combine the images of the sample slide full-scale image after illumination correction in S43 according to the shooting order and position. Finally, obtain the panoramic image of the sample slide full-scale image after illumination correction.

[0068] By adjusting the brightness coefficient α, the brightness of the image after illumination correction can be changed. For example, Figure 5 shows the panoramic image of the full-scale image after illumination correction when α = 1. For example, Figure 6 shows the panoramic image of the full-scale image after illumination correction when α = 1.5. When the value of α is larger, the overall brightness will be higher. At the same time, it can be found that there are no longer streaks caused by uneven illumination in the panoramic image after illumination correction, and the overall observation effect is also clearer.

[0069] A device includes a memory and a processor. The memory stores a computer program. The processor is configured to execute the real-time illumination correction method for the optical microscopic full-scale image in the above embodiments when executing the computer program.

[0070] A storage medium stores a computer program thereon. When the computer program is executed by a processor, it implements the real-time illumination correction method for the optical microscopic full-scale image in the above embodiments.

[0071] Differing from the prior art, the present application provides a real-time illumination correction method, device, and storage medium for an optical microscopic full-scale image. The method includes: S1. Introduce basic information and capture a BGR color image by using a blank glass slide; S2. Introduce a reference value and calculate the average value of pixel values of different color channels of the BGR color image captured by using the blank glass slide; S3. Capture several images of the full-scale image of the sample glass slide; S4. Perform real-time illumination correction on each image of the full-scale image of the sample glass slide in S3. Perform real-time illumination correction on each image of the full-scale image of the sample glass slide in S3 by using the basic information introduced in S1 and the reference value introduced in S2, and finally obtain a panoramic view of the full-scale image of the sample glass slide after illumination correction. The present invention is efficient, stable, and applicable to different devices, and will not affect the effect of illumination correction due to reasons such as attenuation of the optical path system and wear of hardware; it can also perform real-time operations well even on low-computing-power chips.

[0072] The above description shows the main features, basic principles, and advantages of the present invention. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments or examples, and can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, the above embodiments or examples should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, it is intended to include all changes falling within the meaning and scope of the equivalent elements of the claims in the present invention. Any reference signs in the claims should not be regarded as limiting the claimed rights. Any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical principle of the present invention shall fall within the scope of the patent protection of the present invention.

Claims

1. A real-time illumination correction method for a full-volume optical microscopic image, characterized in that: The following steps are involved: S1. Introduce basic information and use a blank glass slide to capture a BGR color image; S2, introduce reference values, and calculate the average values ​​of pixel values ​​of different color channels based on the BGR color image in S1; S3, collecting a plurality of images of the full image of the sample slide; S4, performing illumination correction on each image of the full amount of the sample slide in S3 in real time, performing illumination correction on each image of the full amount of the sample slide in S3 in real time by using the basic information introduced in S1 and the reference value introduced in S2, the steps are as follows: S41, divide the pixel value of the sample image in S3 by the pixel value of the blank glass slide image at the corresponding position in S1, and multiply by the average value of the pixel value of the corresponding channel of the blank glass slide image in S2, S42, convert the decimal obtained in S41 into an integer, S43, truncating the value obtained in S42 and converting it into a pixel value, S44, obtaining the corresponding illumination correction result, combining the images of the full-volume image of the sample slide after illumination correction in S43 according to the shooting order and position, and obtaining a panoramic image of the full-volume image of the sample slide after illumination correction.

2. A real-time illumination correction method for optical microscopic full-volume images as claimed in claim 1, characterized in that: In S1, a blank glass slide is used to capture a BGR color image. The specific steps are as follows: a blank, clean, transparent glass slide is placed under the objective lens of an optical microscope, and a BGR color image under the microscope is captured through a connected camera. Assume that it is Blank, and the resolution is H×W, that is, the height is H, the width is W, and the image size is Blank. uchar (h,w,c) represents the pixel value of the cth channel, hth row, wth column of the image Blank. c=0 represents the blue B channel, c=1 represents the green G channel, and c=2 represents the red R channel. The value is uchar type in computer storage, that is, the value range is 0≤Blank uchar (h,w,c)≤255 and are integers, where 0≤h≤H-1, 0≤w≤W-1, 0≤c≤2, and h, w, and c are all integers.

3. A real-time illumination correction method for optical microscopic full-volume images as claimed in claim 2, characterized in that: In S2, the average values ​​of pixel values ​​of different color channels of the BGR color image captured and collected based on the blank glass slide are calculated using the following formula: in, Represents the average value of all pixel values ​​of the 0th channel, that is, the blue B channel. Represents the average value of all pixel values ​​of the first channel, the green G channel. It represents the average value of all pixel values ​​of the second channel, the red R channel. For the sake of accuracy, the calculation result retains decimals and the computer stores the value as a float type.

4. A real-time illumination correction method for optical microscopic full-volume images as claimed in claim 3, characterized in that: In S4, illumination correction is performed on each image of the full image of the sample slide in real time. The specific steps are as follows: Assume that the collected full image contains a total of N images, and the nth image is assumed to be I (n) , the resolution is consistent with the resolution of the image Blank, Represents the nth image I (n) The pixel value of the cth channel, row, and column is h, assuming that I (n) After illumination correction, it is recorded as O (n) , the specific steps of illumination correction are as follows: S41, construct a three-dimensional array The three-dimensional array has 3 channels with H rows and W columns. The value of the cth channel, hth row and wth column The calculation formula is: in, is the pixel value of the hth row and wth column of the cth channel of the nth sample image, Blank uchar (h,w,c) is the pixel value of the hth row and wth column of the cth channel of the nth blank slide image. is the average value of all pixel values ​​of the cth channel of the blank slide image. The calculation result is retained as a decimal. The computer stores this value as a float type, where 0≤h≤H-1, 0≤w≤W-1, 0≤c≤2, 1≤n≤N, and h, w, c, and n are all integers; S42, construct a three-dimensional array 2 and record it as The three-dimensional array 2 has 3 channels with H rows and W columns. The value of the cth channel, hth row and wth column is The calculation formula is: Among them, the function round() represents rounding a value to an integer, that is, The value is to convert the decimal The value converted to an integer after rounding, the computer stores the value as int type, where 0≤h≤H-1, 0≤w≤W-1, 0≤c≤2, 1≤n≤N, h, w, c, n are all integers; S43, S42 The value is truncated based on 255 and converted into pixel value. The calculation formula for truncation is: in, is the image after illumination correction O (n) The pixel value of the hth row and wth column of the cth channel; S44, obtaining the corresponding illumination correction result, combining the images of the full-volume image of the sample slide after illumination correction in S43 according to the shooting order and position, and finally obtaining a panoramic image of the full-volume image of the sample slide after illumination correction.

5. A real-time illumination correction method for optical microscopic full-volume images as claimed in claim 4, characterized in that: The three-dimensional array constructed in S41 is recorded as The three-dimensional array has 3 channels with H rows and W columns. The value of the cth channel, hth row and wth column The calculation formula is: in, is the pixel value of the hth row and wth column of the cth channel of the nth sample image, Blank uchar (h,w,c) is the pixel value of the hth row and wth column of the cth channel of the nth blank slide image. is the average value of all pixel values ​​of the cth channel of the blank slide image, α is the brightness adjustment coefficient, the calculation result is retained as a decimal, and the computer stores the value as a float type, where 0≤h≤H-1, 0≤w≤W-1, 0≤c≤2, 1≤n≤N, α>0, h, w, c, and n are all integers.

6. A real-time illumination correction method for optical microscopic full-volume images as claimed in claim 2, characterized in that: The resolution is H×W=1200×1920, that is, the height H=1200 and the width W=1920.

7. A method for real-time illumination correction of a full-volume optical microscopic image as claimed in claim 5, characterized in that: α＝1。 8. A real-time illumination correction method for optical microscopic full-volume images as claimed in claim 5, characterized in that: The α=1.

5.

9. A device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images