Pathological image display method and system for aligning visual field of optical microscope

Abstract

The invention relates to a pathological slide processing technology, and discloses a pathological image display method and system for aligning the visual field of an optical microscope, and the method comprises the steps: calculating the real physical size of a visual field range of the microscope under a certain magnification, and obtaining the visual field length under the microscope; calculating the real physical size of the view range of the image software under a certain magnification, and obtaining the view length of the image software under a certain magnification; obtaining an adjustment coefficient, wherein the adjustment coefficient is obtained through the visual field length under the microscope and the visual field length under a certain magnification of image software; and according to the adjustment coefficient, adjusting the pathological image of the image software in the visual field range under a certain magnification. According to the method adopted by the invention, the alignment correction operation only needs to adjust to enable the diameter length of a circle to be equal to the length of the long side of a rectangle, so that the under-mirror visual field is aligned with the software screen display visual field.

Description

Technical Field

[0001] The present invention relates to the technology of pathological slide processing, and particularly to a method and system for displaying pathological images by aligning the fields of view of an optical microscope. Background Art

[0002] In the field of medical pathological images, the digital images of pathological slides are different from other medical images (such as gastroscopes, ultrasounds, CTs, etc.). Their digital images are usually scanned by high-magnification scanners. To ensure that each local detail is close to the display effect under an optical microscope, the image size is much larger than that of traditional images, and the pixel size is generally from 40000*40000 to 240000*240000. With the application of technologies such as digital informatization and artificial intelligence in pathological diagnosis, many pathological image software have entered this field to assist doctors in their daily work of reading slides and remote consultations, greatly improving the efficiency of pathological diagnosis and improving the current situation where the demand for pathological diagnosis cannot be met in China.

[0003] During the process of assisting with the application of pathological image software, one of the most important functions is the preview of electronic images. This function enables doctors to get rid of mechanical repetitive operations such as loading and unloading slides, adjusting the field of view, and adjusting the focus using a microscope. They can directly perform unified and batch electronic reading of slides on a computer screen, and even assist in reading slides with the help of the image analysis functions built into some pathological image software, greatly improving the reading efficiency and reducing the labor intensity of doctors.

[0004] In the application process of this digital transformation, the most important basis is that the display effect of digital pathological images at various magnifications should be as consistent as possible with the imaging effect under a traditional optical microscope. It is necessary to ensure that the doctor's reading and interpretation results are not affected by additional imaging effect differences. Usually, under a traditional microscope, doctors will quickly view the whole slide with a large field of view under 4X / 10X objectives. When local abnormal lesions are found, they will use a 20X objective for magnified observation. When examining detailed lesions, they will switch to 40X or even 80X objectives for careful confirmation, and finally give an interpretation opinion. In existing pathological image software, although the implementation of this function is basically the same as the principle under an optical microscope, after reading the positive film image, the software provides fixed magnified fields of view of 4X / 10X / 20X / 40X / 80X for doctors to view. However, its magnification principle is the absolute magnification of image pixels. For example, for a slide with a pixel size of 100000*100000, after clicking the 20X display button, through absolute pixel magnification, the image in the picture is displayed as an area of 10W pixels after 20-fold magnification, that is, an area of 5000*5000. This traditional technology basically meets the doctor's need to view images at different magnifications during the process of reading slides. Summary of the Invention

[0005] In view of the obvious scale distortion problem of cross-modal image data in the prior art, that is, the field of view sizes at different magnifications in the pathological image software are inconsistent with those under the objective lenses of different magnifications of the microscope, the present invention provides a method and system for displaying pathological images that align the fields of view of an optical microscope.

[0006] To solve the above technical problems, the present invention is solved by the following technical solutions:

[0007] A method for displaying a pathological image software that aligns the fields of view of an optical microscope, the method comprising:

[0008] Calculating the true physical size of the field of view range at a certain magnification of the microscope to obtain the field of view length under the microscope;

[0009] Calculating the true physical size of the field of view range at a certain magnification of the image software to obtain the field of view length at a certain magnification of the image software;

[0010] Obtaining an adjustment coefficient by using the field of view length under the microscope and the field of view length at a certain magnification of the image software;

[0011] Adjusting the pathological image of the field of view range at a certain magnification of the image software according to the adjustment coefficient.

[0012] Preferably, calculating the true physical size of the field of view range at a certain magnification of the microscope to obtain the field of view length under the microscope includes:

[0013] Using the upper-mounted camera of the microscope to take a picture of the field of view and save the screenshot; reading the total length of the scale on the microscale glass slide as D, and obtaining the pixel length of the picture collected by the microscope camera as N; using the pixel distance measurement algorithm to measure the number of pixels between the left and right scale lines of the microscale in the picture as P ref ;

[0014] Calculating the field of view length L1 under the microscope according to the obtained number of pixels

[0015] L1 = D * N / P ref ;

[0016] Wherein, D is the total length of the scale on the microscale glass slide, N is the pixel length of the picture collected by the microscope camera, and P ref is the number of pixels between the left and right scale lines of the microscale.

[0017] Preferably, the obtaining of the number of pixels P ref between the left and right scale lines of the microscale includes:

[0018] Reading the image, and using the microscale glass slide under the microscope to assist in obtaining the pathological image of the field of view under the microscope;

[0019] Gray conversion of pathological images, performing gray conversion on the obtained pathological images;

[0020] Edge detection, performing edge detection on the gray-converted pathological images and returning the horizontal axis X coordinates of all pixels of the scale line contour edge;

[0021] Obtaining the maximum spacing of the scale line contour on the X axis;

[0022] Calculating the number of pixels P ref , using the largest X among the coordinate values of all pixels max , subtracting the smallest X among the coordinate values of all pixels min , thereby obtaining the number of pixels P ref

[0023] P ref =X max -X min

[0024] wherein, X max is the maximum coordinate value of the X axis in the contour line pixel set, and X min is the minimum coordinate value of the X axis in the contour line pixel set.

[0025] Preferably, calculating the true physical size of the field of view at a certain magnification of the image software, and obtaining the field of view length at a certain magnification of the image software includes:

[0026] Obtaining that the pixel length of the WSI image displayed in the canvas is W under the 20x setting of the pathological image software; 20x ; obtaining the scanner scanning resolution parameter for generating the WSI image as S res ; calculating the physical length size of the glass slide field of view displayed by the software as L2, L2 = W 20x / S res ;

[0027] wherein, W 20x is the pixel length of the WSI image, and S res is the scanner scanning resolution parameter.

[0028] Preferably, obtaining the adjustment coefficient, obtaining the adjustment coefficient C through the field of view length under the microscope and the field of view length at a certain magnification of the image software coeff , C coeff = field of view size under the microscope / field of view size under the image software;

[0029]

[0030] wherein, D is the total length of the scale on the macro scale glass slide; N is the pixel length of the camera screen under the microscope;

[0031] α is the thermal offset coefficient of the microscope camera, β is the thermal offset coefficient of the scanner lens, ΔT1 is the environmental temperature offset of the microscope; ΔT2 is the environmental temperature offset of the scanner; P ref is the number of pixels corresponding to the total length of the microscopic field macroruler; W kx The pixel length of the WSI image displayed at k times the software; The pixel height of the WSI image displayed at k times the software.

[0032] Preferably, according to the adjustment coefficient, the adjustment of the pathological image in the field of view range of the image software at a certain magnification includes: through the correction coefficient C coeff , C coeff *W kx Calculate the finally corrected WSI display pixel size at each k times and the WSI display pixel sizes W' at all other magnifications nx , H' nx ; where, W' nx is the pixel length of the WSI image displayed at n times after update; H' nx is the pixel height of the WSI image displayed at n times after update.

[0033] To solve the above technical problems, the present invention also provides a pathological image display system for aligning the field of view of an optical microscope, which includes:

[0034] A microscopic field length acquisition module, which calculates the true physical size of the field of view range of the microscope at a certain magnification and obtains the microscopic field length;

[0035] An image software field length acquisition module at a certain magnification, which calculates the true physical size of the field of view range of the image software at a certain magnification and obtains the field length of the image software at a certain magnification;

[0036] An adjustment coefficient acquisition module, which obtains the adjustment coefficient by the microscopic field length and the field length of the image software at a certain magnification;

[0037] A pathological image module, which adjusts the pathological image in the field of view range of the image software at a certain magnification according to the adjustment coefficient.

[0038] Since the present invention adopts the above technical solutions, it has remarkable technical effects:

[0039] In the present invention, since the microscopic field of view is a circular area and the image software displays the area on the screen as a rectangle. Therefore, the alignment and correction operation only needs to be adjusted so that the circular diameter length is equal to the long side length of the rectangle to ensure the alignment of the microscopic field of view and the software screen display field of view. At the same time, the conventional objective lenses of the microscope are magnifications of 4x, 10x, 20x, 40x, 80x. During the correction process of this method, the field of view range under the 20x objective lens is used as the reference for calculation, and the correction result is finally converted to all other magnifications. Brief Description of the Drawings

[0040] Figure 1 is a schematic flow diagram of the present invention.

[0041] Figure 2 is a microscale slide diagram, which is an auxiliary measurement tool used in the calculation process of the present invention.

[0042] Figure 3 is a microscale field of view image captured by the camera above the microscope during the image acquisition process of the present invention.

[0043] Figure 4 is the image of the physical slide seen under the 20x eyepiece of the microscope (left) and the display image of the scanned image of the same physical slide shown by the pathological image software at the 20x setting at the same position (right) without using the method of the present invention.

[0044] Figure 5 is the image of the physical slide seen under the 20x eyepiece of the microscope (left) and the display image of the scanned image of the same physical slide shown by the pathological image software at the 20x setting at the same position (right) when using the present invention. Detailed Description of the Preferred Embodiments

[0045] The present invention will be further described in detail below with reference to the drawings and embodiments.

[0046] Embodiment

[0047] First, use a microscale slide under the microscope to assist in obtaining the actual physical size of the microscopic field of view. Adjust the microscope to the 20x objective lens and place the microscale slide on the stage. Use the camera on the microscope to capture the image of the field of view and save the screenshot. Read the total length of the scale on the microscale slide as D, and obtain the pixel length of the image captured by the microscope camera as N. Use the pixel distance measurement algorithm to measure the number of pixels between the left and right scale lines of the microscale in the image as P ref ; Pixel distance measurement algorithm.

[0048] After the above parameters, the physical size length of the microscopic field of view can be calculated as micrometers;

[0049] Obtain the pixel length of the WSI image displayed in the canvas of the pathological image software at the 20x setting as W 20x ; Obtain the scanning resolution parameter of the scanner that generates the WSI image as S res ; After combining the above parameters, the physical length size of the slide field of view displayed by the software can be calculated as micrometers.

[0050] Combine the physical size lengths respectively displayed in the microscopic view and the software page to calculate the correction coefficient At the same time, since the acquisition of each field of view image comes from the camera chips in their respective hardware, and the imaging chip will produce dark current due to reasons such as thermal excitation of the sensor in different temperature environments, the dark current will cause noise and pixel distortion in the image, and the distortion will cause pixel-level deviation, thus resulting in a deviation of micrometers in image ranging. The greater the difference between the temperature and the factory calibration temperature of the camera, the more obvious the ranging deviation. Based on this, a temperature compensation calculation factor is introduced in this method, and the calculation method is: field of view size × (1 + α·ΔT), where α is the thermal offset coefficient of the camera chip (this parameter is provided by the manufacturer), and ΔT is the difference between the environmental temperature and the reference temperature of the camera. Based on the above information, the correction coefficient for the 20x objective can be updated and calculated as C coeff , and the correction coefficients for the 4x, 10x, 20x, 40x, and 80x objectives are extended to: 5·C coeff 、2·C coeff 、C coeff 、

[0051] Through the correction coefficient C coeff , W 20x ·C coeff Calculate the finally corrected WSI display pixel size at 20 times and the WSI display pixel sizes W' nx 、H' nx .

[0052] Finally, integrate and summarize the above calculation steps. The calculation formula for the correction coefficient C coeff is as follows in formula (1.1), and the calculation formula for the finally corrected display size at each magnification of the software is as follows:

[0053]

[0054] Calculate the correction coefficient for the 20x setting and extend it to the correction coefficient matrices for the 4x, 10x, 20x, 40x, and 80x settings as: Multiply by the reference size to obtain the finally updated WSI display size at each magnification setting: (W' 4x H' 4x ),(W' 10x H' 10x ),(W' 20x H' 20x ),(W' 40x H' 40x ),(W' 80x H' 80x ).

[0055] Table 1, Parameter Definition Table

[0056]

[0057]

[0058] First, place the microscale glass slide, such as Figure 2 under the 20x lens of the microscope. After adjusting the field of view area so that the scale distance area of the glass slide can be completely observed in the microscopic image, use the microscope camera to save the image within the field of view. Figure 3 . The pixel length of this image is the pixel length N of the camera image under the microscope, which is 1920 pixels. For Figure 3 using the pixel distance measurement algorithm to detect the pixel size P of the maximum distance between the left and right ends of the microscale. ref It is 1673 pixels. Then, according to the scale on the microscale glass slide, the distance between each dividing line is 10 μm, and there are 100 intervals in total, so D is known to be 1000 μm. Finally, the thermal offset coefficient of the microscope camera is provided by the manufacturer, α is 2.3×10-6 / °C, and the working environment temperature is 20°.

[0059] Under the 20x display settings of the image software, the default pixel size of the WSI image area displayed is 4000*3000 pixels, that is, W 20x is 4000 pixels, and H 20x is 3000 pixels. At the same time, the resolution of the example scanner is 0.5 μm / pixel, that is, S res is 0.5. The thermal offset coefficient of the scanner lens is provided by the manufacturer, β is 1.7×10 -6 / °C, and the working environment temperature is 20°.

[0060] Calculate the correction coefficient

[0061] The correction coefficient matrix expanded for other magnifications is: [2.869 1.1476 0.5738 0.2869 0.14345]

[0063] Calculate the final display image size. The WSI image sizes displayed by the software at each magnification after correction are:

[0064]

[0065] Finally, the WSI image area displayed by the software at the 4x setting is 11476*8607 pixels, at the 10x setting is 4590*3443 pixels, at the 20x setting is 2295*1721 pixels, at the 40x setting is 1148*861 pixels, and at the 80x setting is 574*430 pixels.

Claims

1. A software display method for aligning pathological images in the field of view of an optical microscope, the method comprising: Calculating the true physical size of the field of view at a certain magnification of the microscope to obtain the field of view length under the microscope; Calculating the true physical size of the field of view at a certain magnification of the image software to obtain the field of view length at a certain magnification of the image software; Obtaining the adjustment coefficient, and obtaining the adjustment coefficient through the field of view length under the microscope and the field of view length at a certain magnification of the image software; Adjusting the pathological image of the field of view at a certain magnification of the image software according to the adjustment coefficient.

2. The method for displaying a pathological image for aligning the field of view of an optical microscope according to claim 1, wherein Calculating the true physical size of the field of view at a certain magnification of the microscope to obtain the field of view length under the microscope includes: Use the overhead camera on the microscope to capture the image in the field of view and save the screenshot; read the total length of the scale on the micrometer glass as D, and obtain the pixel length of the image collected by the microscope camera as N; use the pixel ranging algorithm to measure the number of pixels between the left and right scale lines of the micrometer in the image as ; Calculating the field of view length L1 under the microscope according to the obtained number of pixels ； where D is the total length of the scale on the macro ruler glass piece, and N is the pixel length of the image collected by the microscope camera, is the number of pixels between the left and right scale lines of the macro ruler.

3. A method for displaying a pathological image of an aligned optical microscope field of view according to claim 2, characterized in that, The number of pixels between the left and right scale lines of the macro ruler is obtained as follows: Reading the image, and obtaining the pathological image of the field of view under the microscope by using a macroscale slide under the microscope; Gray conversion of the pathological image, and performing gray conversion on the obtained pathological image; Edge detection, performing edge detection on the gray-converted pathological image, and returning the horizontal axis X coordinate values of all pixels of the scale line contour edge; Obtaining the maximum distance of the scale line contour on the X axis; Calculate the number of pixels , use the one with the largest coordinate value among all pixels , subtract the one with the smallest coordinate value among all pixels , thereby obtaining the number of pixels : ； Among them, is the maximum coordinate value of the X-axis in the set of contour line pixels, is the minimum coordinate value of the X-axis in the set of contour line pixels.

4. A method for displaying a pathological image by aligning the field of view of an optical microscope according to claim 1, characterized in that, Calculating the true physical size of the field of view at a certain magnification of the image software to obtain the field of view length at a certain magnification of the image software includes: When the pathological image acquisition software is set to 20x, the pixel length of the WSI image displayed in the canvas is ; the scanning resolution parameter of the scanner that generates the WSI image is ; the calculated physical length dimension of the glass slide field of view displayed by the software is L2, ; Among them, is the pixel length of the WSI image, is the scanner scanning resolution parameter.

5. A method for displaying a pathological image of an aligned optical microscope field of view according to claim 1, characterized in that, Obtaining the adjustment coefficient: The adjustment coefficient is obtained by the field of view length under the microscope and the field of view length at a certain magnification of the image software. , = Field of view size under the microscope / Field of view size under the image software; ； Wherein, D is the total length of the scale on the microscale glass piece; N is the pixel length of the camera image under the microscope; α is the thermal offset coefficient of the microscope camera, and β is the thermal offset coefficient of the scanner lens, is the microscope ambient temperature offset; is the scanner ambient temperature offset; is the pixel amount corresponding to the total length of the microscale in the microscope field of view; The pixel length of the WSI image displayed at k times of the software; the pixel height of the WSI image displayed at k times of the software.

6. A method for software display of pathological images for aligning the field of view of an optical microscope, as claimed in claim 1, wherein Adjusting the pathological images within the field of view at a certain magnification of the image software according to the adjustment coefficient includes: through the correction coefficient , calculating the finally corrected WSI display pixel size at each k - fold and the WSI display pixel sizes at all other magnifications , ; where is the pixel length of the WSI image displayed at the updated n - fold; is the pixel height of the WSI image displayed at the updated n - fold.

7. A pathological image display system for aligning the fields of view of an optical microscope, characterized in that, Including: A module for obtaining the field of view length under the microscope, calculating the true physical size of the field of view at a certain magnification of the microscope to obtain the field of view length under the microscope; A module for obtaining the field of view length at a certain magnification of the image software, calculating the true physical size of the field of view at a certain magnification of the image software to obtain the field of view length at a certain magnification of the image software; A module for obtaining the adjustment coefficient, obtaining the adjustment coefficient through the field of view length under the microscope and the field of view length at a certain magnification of the image software; A pathological image module, adjusting the pathological image of the field of view at a certain magnification of the image software according to the adjustment coefficient.

Images