Digitalization Methods and Systems for Low-Resolution Imaging of Pathological Sections

Abstract

This application provides a method and system for digitizing pathological slides using low-resolution imaging. The method includes the following steps: imaging a multi-resolution calibration template using both low-resolution and high-resolution imaging systems to acquire low-resolution and high-resolution images; constructing high / low-resolution image pairs of pathological tissues using information provided by the multi-resolution calibration template; training a super-resolution image reconstruction algorithm in a supervised manner using multiple high / low-resolution image pairs to establish a computational model; imaging the pathological slides using the low-resolution imaging system, and then converting the low-resolution images into high-resolution images using the computational model. Through low-resolution imaging and computational model conversion, a new digitization technique for pathological slides based on low-resolution imaging is established, solving the problems of slow speed, local defocusing, and geometric distortion in current high-resolution imaging digitization, saving digitization time, and improving image quality.

Description

Technical Field

[0001] This invention relates to the field of pathological imaging technology, and in particular to a method and system for digitizing pathological sections using low-resolution imaging. Background Technology

[0002] The digitization of pathological slides is the foundation of digital pathology, and its principle is optical imaging technology. Current slide digitization processes involve using a high-magnification lens, such as a 40x lens, to precisely focus on and magnify the pathological slide. The magnified slide is then projected onto a photosensitive device, and after analog-to-digital conversion, a digital signal is acquired. Through quantization processing, a high-resolution full-section image of the pathological slide is obtained.

[0003] However, the main bottlenecks of high-resolution imaging digitization technology for pathological slides are: First, high-magnification lenses have extremely shallow depth of field, thus requiring a high degree of flatness of the human tissue on the slide, easily causing local defocusing and blurring. Second, due to the difficulty in ensuring that every slide is flat during batch production, multiple local locations on the slide need to be focused or repeatedly scanned to obtain a clear full-slide image, resulting in a long scanning time. Third, high-magnification lenses are more likely to exhibit geometric distortion, causing changes in cell morphology in the image. Fourth, imaging devices based on high-magnification lenses and matching photosensitive devices are expensive, resulting in a pathology scanner costing over a million dollars, making it difficult for medical institutions to afford. In summary, current high-resolution imaging digitization technology for pathology is expensive, has high requirements for specimens, is slow in scanning speed, and is prone to defocusing, making it difficult to economically and reliably convert pathological slides into full-slide images. Summary of the Invention

[0004] This application provides a method and system for digitizing pathological sections using low-resolution imaging, thereby improving the imaging effect.

[0005] In a first aspect, this application provides a method for digitizing pathological sections using low-resolution imaging, the method comprising the following steps:

[0006] The multi-resolution calibration template was imaged using both a low-resolution imaging system and a high-resolution imaging system; low-resolution and high-resolution images of the pathological tissue, color, geometric morphology, and positioning information of the multi-resolution calibration template were acquired;

[0007] Based on the color, geometric shape, and positioning information of the multi-resolution calibration template, the low-resolution and high-resolution images of the pathological tissue are preprocessed and corrected to obtain high / low resolution image pairs of pathological tissue with color normalization, geometric distortion correction, and pixel alignment.

[0008] A super-resolution image reconstruction algorithm is trained in a supervised manner using multiple high / low resolution image pairs to establish a computational model that can reconstruct low-resolution images of pathological tissues into high-resolution images.

[0009] A low-resolution imaging system is used to image pathological slides, and then the computational model is used to convert the low-resolution images into high-resolution images.

[0010] In the above technical solution, by employing a multi-resolution calibration template, high / low resolution image pairs with corrected color and geometric distortion, and precise pixel alignment, were obtained, achieving an advancement in accurate training of the computational model. Through low-resolution imaging and computational model conversion, a new digital pathological slide technique based on low-resolution imaging was established, solving the problems of slow speed, local defocusing, and geometric distortion in current high-resolution imaging digital slides, saving digitization time and improving image quality.

[0011] In one specific implementation scheme, the calibration template includes: a color calibration area, a multi-resolution geometric distortion calibration area, a multi-resolution translation calibration area, a multi-resolution rotation calibration area, and a slice area; wherein:

[0012] The color calibration area and the multi-resolution geometric distortion calibration area are used to provide color distribution and geometric distortion of the high-resolution imaging system and the low-resolution imaging system; the multi-resolution translation calibration area and the multi-resolution rotation calibration area provide the relative translation and rotation of the high / low resolution images; the slicing area is used to place different pathological slices.

[0013] In one specific implementation, the use of a low-resolution imaging system and a high-resolution imaging system to image the multi-resolution calibration template specifically involves:

[0014] The low-resolution imaging system obtains a low-resolution image of the multi-resolution calibration template by scanning a certain number of image blocks of the multi-resolution calibration template and stitching them together.

[0015] The high-resolution imaging system obtains a high-resolution image of the multi-resolution calibration template by scanning a certain number of image blocks of the multi-resolution calibration template and stitching them together.

[0016] In one specific implementation, the geometric distortion, translation, and rotation marker information on the multi-resolution calibration template image is a combination of multi-resolution line segments. A specific resolution line segment presents a single pixel width in the high / low resolution image, and the center lines of the multi-resolution line segments coincide, thereby obtaining sub-pixel level positioning accuracy.

[0017] In one specific implementation scheme, based on the color, geometric shape, and positioning information of the multi-resolution calibration template, preprocessing image correction is performed on the low-resolution and high-resolution images of the pathological tissue to obtain high / low-resolution image pairs of pathological tissue with color normalization, geometric distortion correction, and pixel alignment; specifically:

[0018] Using the geometric distortion, color, translation, and rotation marker information on the image on the multi-resolution calibration template, the pixel relationship of high / low resolution images of pathological tissue in the slice area is established.

[0019] In one specific implementation, the color calibration area is used to provide the color distribution of the imaging system, specifically including:

[0020] By measuring the color changes in the color calibration area, the color distribution of low-resolution and high-resolution images and their correspondence with real colors are obtained, thereby achieving color standardization.

[0021] In one specific implementation, obtaining high / low resolution image pairs of pathological tissue with geometric distortion correction and pixel alignment specifically includes:

[0022] The deformation of pixels in high-resolution and low-resolution images is obtained through a multi-resolution geometric distortion calibration region.

[0023] The relative rotation angle of pixels in high-resolution and low-resolution images is obtained through a multi-resolution rotation calibration area;

[0024] The relative translation amount of pixels in high-resolution and low-resolution images is obtained by using a multi-resolution translation calibration area.

[0025] In one specific implementation, obtaining the relative rotation angle of pixels in the high-resolution image and the low-resolution image through a multi-resolution rotation calibration area, and obtaining the relative translation amount of pixels in the high-resolution image and the low-resolution image through a multi-resolution translation calibration area, specifically involves:

[0026] The deformation of pixels in high-resolution and low-resolution images is obtained by using multi-resolution specific markers in the multi-resolution geometric distortion calibration region; the translation and rotation of pixels in high-resolution and low-resolution images are obtained by using multi-resolution specific markers in the multi-resolution translation calibration region and the multi-resolution rotation calibration region; a nonlinear rigid body transformation matrix is ​​constructed to achieve pixel position alignment of high / low resolution images; and a high / low resolution image pair is established through color normalization, geometric distortion correction, and pixel alignment.

[0027] In one specific implementation, the step of training the super-resolution image reconstruction algorithm in a supervised manner using multiple high / low resolution image pairs specifically includes:

[0028] The training objective function ensures that the high-resolution images generated from the low-resolution images are highly similar to the pairs of high-resolution images, including similarities in pixels, colors, textures, and perceptual features.

[0029] Secondly, a low-resolution imaging digitization system for pathological slides is provided. The system includes a low-resolution imaging system comprising: a low-magnification lens, optoelectronic devices, a signal processing unit, an image correction unit, and a parallel computing unit.

[0030] The low-magnification lens is used to scan pathological sections to obtain corresponding low-magnification optical magnified images; the optoelectronic device is used to convert the optical magnified images into analog electrical signals; the signal processing unit is used to perform analog-to-digital conversion on the analog electrical signals to obtain low-resolution images; the image correction unit is used to correct image color and pixel distortion; the parallel computing unit is used to convert the low-resolution images into high-resolution images according to the established computing model that can reconstruct low-resolution images of pathological tissues into high-resolution images.

[0031] In the above technical solution, by employing a multi-resolution calibration template, high / low resolution image pairs with corrected color and geometric distortion, and precise pixel alignment, were obtained, achieving an advancement in accurate training of the computational model. Through low-resolution imaging and computational model conversion, a new digital pathological slide technique based on low-resolution imaging was established, solving the problems of slow speed, local defocusing, and geometric distortion in current high-resolution imaging digital slides, saving digitization time and improving image quality. Attached Figure Description

[0032] Figure 1 A flowchart of the low-resolution imaging digitization method for pathological sections provided by the present invention.

[0033] Figure 2 This is a structural block diagram of the low-resolution imaging system provided by the present invention.

[0034] Figure 3 The structural block diagram of the high-resolution imaging system provided by the present invention.

[0035] Figure 4 This is a structural block diagram of the multi-resolution calibration template provided in the embodiments of this application.

[0036] Figure 5 This is a schematic diagram of the structure of each part of the multi-resolution calibration template provided by the present invention.

[0037] Figure 6 This is a schematic diagram illustrating the image distortion amount reflected in the multi-resolution geometric distortion calibration region provided by the present invention.

[0038] Figure 7 The color transformation lookup table provided for this invention.

[0039] Figure 8 This is a schematic diagram of the multi-resolution marking information of the multi-resolution geometric distortion calibration region provided by the present invention.

[0040] Figure 9 This is a schematic diagram of the multi-resolution marker information for the multi-resolution translation and multi-resolution rotation calibration area provided by the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the application will now be described in further detail with reference to the accompanying drawings.

[0042] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of this specification should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar words used in one or more embodiments of this specification do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0043] The principle of the low-resolution imaging digitization technology proposed in this invention is to magnify pathological slides using a low-magnification lens, then project the image onto a low-resolution photosensitive device. After analog-to-digital conversion, a low-resolution full-slide image is obtained through quantization, and finally converted into a high-resolution full-slide image through a computational model. Low-resolution lenses have the advantage of a large depth of field, thus reducing the high requirements for the flatness of the human tissue slides, making them more suitable for use in primary healthcare units with limited slide preparation capabilities. Secondly, multiple refocusing is unnecessary, greatly accelerating the scanning speed. Thirdly, low-resolution lenses are easy to manufacture and have minimal geometric distortion. Fourthly, the pathological slide technology and device based on low-resolution imaging are inexpensive, have fast scanning speeds, and lower requirements for slide quality, achieving a balance between economy and efficiency, and have broader application prospects.

[0044] While digital technologies and methods for low-resolution imaging offer many advantages, their main technical bottleneck lies in obtaining a computational model for accurately reconstructing high-resolution pathological whole-slice images from low-resolution images. Current techniques for reconstructing high-resolution images, represented by super-resolution image processing, have made significant progress in many fields. These techniques are primarily based on supervised learning deep neural networks, and their reconstruction quality heavily depends on the quantity and quality of the paired low-resolution and high-resolution images.

[0045] However, when the slide is imaged twice using lenses with different magnifications, the resulting low-resolution and high-resolution images exhibit color differences and their pixel coordinates are not precisely aligned. One technique involves image registration to transform the low-resolution image coordinates to the high-resolution image coordinates. Due to the differences in color and resolution between the two images, as well as geometric distortions, image registration struggles to accurately extract corresponding spatial features from the two images, resulting in large registration errors. Another technique involves first acquiring a high-resolution image and then downsampling it to a low-resolution image, thus obtaining a high / low-resolution image pair. The drawback of this method is that the low-resolution image used for training the model is not obtained through the imaging method, leading to a significant decrease in image quality when reconstructing from the actual low-resolution image. Pathological diagnosis heavily relies on the accurate reconstruction of cell morphology and tissue structure; if the reconstructed high-resolution image is of poor quality, it lacks clinical application value. Therefore, this application provides a method for digitizing pathological slides using low-resolution imaging to improve imaging results. The following detailed description, in conjunction with specific figures and embodiments, further clarifies this method.

[0046] refer to Figure 1 , Figure 1 A flowchart of a method for digitizing pathological sections using low-resolution imaging, as provided in an embodiment of this application, is shown.

[0047] The method provided in this application includes the following steps:

[0048] Step 001: Image the multi-resolution calibration template using both low-resolution and high-resolution imaging systems; acquire low-resolution and high-resolution images of the pathological tissue, color, geometric morphology, and localization information of the multi-resolution calibration template:

[0049] Specifically, the low-resolution imaging system and high-resolution imaging system provided in this application are systems with different resolutions. For example... Figure 2The diagram shows the structural block of the low-resolution imaging system provided in this application embodiment. Specifically, it includes a low-magnification lens, optoelectronic devices, a signal processing unit, an image correction unit, and a parallel computing unit, which outputs a low-resolution image of a pathological slide. In use, the low-magnification lens scans the pathological slide to obtain a corresponding low-magnification optically magnified image. The optoelectronic devices convert the optically magnified image into an analog electrical signal. The signal processing unit performs analog-to-digital conversion, quantization, and other processing steps on the electrical signal to obtain the low-resolution original image. The image correction unit is used to correct image color and geometric distortion; the parallel computing unit reconstructs a high-resolution image from a low-resolution image.

[0050] As an alternative, the low magnification is set to 5x to accommodate the pixel size of the smallest cells in the image for pathological diagnosis.

[0051] like Figure 3 As shown, the high-resolution imaging system includes a high-magnification lens, optoelectronic devices, a signal processing unit, and image correction. The high-magnification lens scans pathological sections to obtain a corresponding high-magnification optically magnified image. The optoelectronic devices convert the optically magnified image into an analog electrical signal. The signal processing unit performs analog-to-digital conversion, quantization, and other processing steps on the electrical signal to obtain a high-resolution image. When using a high-resolution imaging system, the process also includes focusing on a multi-resolution calibration template using a focusing algorithm to obtain a clear, high-resolution image. Image correction is used to correct image color and geometric distortion.

[0052] As an optional feature, the high magnification is set to 40x to display the cellular and tissue structural details of pathological sections with high definition.

[0053] refer to Figure 4 As shown, the multi-resolution calibration template is designed as a combination of calibration information areas, including a slice area, a multi-resolution geometric distortion calibration area, a color calibration area, a multi-resolution translation calibration area, and a multi-resolution rotation calibration area. Each calibration area contains marker information for geometric distortion, color, translation, and rotation evaluation, providing the calibration information required for image correction of high / low resolution images. An exemplary multi-resolution calibration template includes: a color calibration area, a multi-resolution geometric distortion calibration area, a multi-resolution translation calibration area, a multi-resolution rotation calibration area, and a slice area; wherein, the color calibration area and the multi-resolution geometric distortion calibration area are used to provide the color distribution and geometric distortion of the imaging system; the multi-resolution translation calibration area and the multi-resolution rotation calibration area provide the relative translation and relative rotation amounts of high-resolution and low-resolution images; the slice area is used to place different pathological slices.

[0054] When using low-resolution imaging systems and high-resolution imaging systems to image a multi-resolution calibration template, the specific process is as follows: the low-resolution imaging system obtains a low-resolution image by scanning a certain number of image blocks of the multi-resolution calibration template and stitching them together; the high-resolution imaging system obtains a high-resolution image by scanning a certain number of image blocks of the multi-resolution calibration template and stitching them together.

[0055] As an alternative, the multi-resolution geometric distortion calibration area is made of a transparent material and covers the slice area so that it can be imaged in the same spatial coordinates as the slice.

[0056] Step 002: Based on the color, geometric shape and positioning information of the multi-resolution calibration template, perform preprocessing image correction on the low-resolution and high-resolution images of pathological tissues to obtain high / low resolution image pairs of pathological tissues with color normalization, geometric distortion correction and pixel alignment.

[0057] Specifically, when forming image pairs, the pixel relationships of high / low resolution images of pathological tissue on the slice area are established using geometric distortion, color, translation, and rotation markers on the calibration template image. Specifically: the deformation of pixels in the high-resolution and low-resolution images is obtained; the relative rotation angle of pixels in the high-resolution and low-resolution images is obtained; the relative translation of pixels in the high-resolution and low-resolution images is obtained; thus, based on the pixel deformation, relative rotation angle, and relative translation, the image pairs of high / low resolution pathological tissue for geometric distortion correction and pixel alignment are determined. Specifically, the deformation of pixels in the high-resolution and low-resolution images is obtained through multi-resolution specific markers in the multi-resolution geometric distortion calibration area; the translation and rotation of pixels in the high-resolution and low-resolution images are obtained through multi-resolution specific markers in the multi-resolution translation calibration area and the multi-resolution rotation calibration area, constructing a nonlinear rigid body transformation matrix to achieve pixel alignment of the high / low resolution images.

[0058] Furthermore, by analyzing color changes in the color calibration region, the color distribution of low-resolution and high-resolution images and their correspondence with true colors are obtained. This allows for the acquisition of high / low-resolution image pairs of pathological tissues that have undergone color normalization, geometric distortion correction, and pixel alignment.

[0059] Step 003: Train the super-resolution image reconstruction algorithm in a supervised manner using multiple high / low resolution image pairs to establish a computational model that can reconstruct low-resolution images of pathological tissues into high-resolution images;

[0060] Specifically, the computational model satisfies the following during training: the high-resolution image generated from the low-resolution image and the paired high-resolution images have high similarity, including similarity in pixels, colors, textures, and perceptual features.

[0061] Step 004: Use a low-resolution imaging system to image the pathological slides, and then use a computational model to convert the low-resolution images into high-resolution images.

[0062] In the above technical solution, by employing a multi-resolution calibration template, high / low resolution image pairs with corrected color and geometric distortion, and precise pixel alignment, were obtained, achieving an advancement in accurate training of the computational model. Through low-resolution imaging and computational model conversion, a new digital pathological slide technique based on low-resolution imaging was established, solving the problems of slow speed, local defocusing, and geometric distortion in current high-resolution imaging digital slides, saving digitization time and improving image quality.

[0063] To facilitate understanding of the above solutions provided in the embodiments of this application, the steps of the above methods are described in detail below.

[0064] This application first introduces the system used in the low-resolution imaging method for digitizing pathological sections. The system includes a low-resolution imaging system and a calibration system. The low-resolution imaging system includes a low-magnification lens, optoelectronic devices, a signal processing unit, an image correction unit, and a parallel computing unit. This low-resolution imaging system outputs both low-resolution and high-resolution images of the pathological sections.

[0065] First, a low-magnification lens is used to scan the pathological slide to obtain a corresponding low-magnification optical image. Optoelectronic devices convert the optical image into an analog electrical signal. A signal processing unit then performs analog-to-digital conversion and quantization on the electrical signal to obtain a low-resolution original image. After image correction for color and geometric distortion, as well as translation and rotation, the corrected low-resolution image I is obtained. LR .

[0066] Preferably, the low magnification is set to 5x to accommodate the pixel size of the smallest cell on the pathological slide in the image.

[0067] The aforementioned calibration system includes a high-resolution imaging system and a multi-resolution calibration template. The high-resolution imaging system includes a high-magnification lens, optoelectronic devices, a signal processing unit, and an image correction unit.

[0068] Preferably, high-magnification lenses use a magnification of 20x or higher.

[0069] Preferably, the signal processing unit performs analog-to-digital conversion, quantization, and other processing steps on the electrical signal.

[0070] Preferably, the signal processing unit also includes a focusing algorithm to enable the lens to focus on the multi-resolution calibration template to obtain a clear, high-resolution image.

[0071] Preferably, the focusing algorithm analyzes the sharpness of the image edges and fine-tunes the object distance from the lens to the multi-resolution calibration template based on the sharpness to obtain the ideal sharpness.

[0072] Pathological sections are scanned using a high-magnification lens to obtain a corresponding high-magnification optical image. Optoelectronic devices convert the optical image into an analog electrical signal, and a signal processing unit performs analog-to-digital conversion, quantization, and other processing steps on the electrical signal to obtain a high-resolution image. After color correction, geometric distortion correction, and translation / rotation image correction, the corrected high-resolution image I is obtained. HR .

[0073] The aforementioned multi-resolution calibration template is designed as a combination of calibration information regions, including a slice area, a multi-resolution geometric distortion calibration area, a color calibration area, a multi-resolution translation calibration area, and a multi-resolution rotation calibration area. Each calibration area contains marker information for geometric distortion, color, translation, and rotation evaluation, providing the calibration information required for image correction of high / low resolution images.

[0074] Preferably, each area of ​​the multi-resolution calibration template is designed to be movable and can be moved. One or more calibration areas can be placed.

[0075] Preferably, the slicing area is where pathological slides are placed, allowing for the removal and placement of different pathological slides. The other calibration areas remain in their original positions when changing slides or during imaging.

[0076] Preferably, the marking information of each region on the multi-resolution calibration template is as follows: Figure 5 As shown.

[0077] Preferably, the multi-resolution geometric distortion calibration region is a parallel line grid, and each grid intersection is used to reflect the geometric distortion displacement of the image position where the grid point is located.

[0078] Preferably, the multi-resolution geometric distortion calibration area is made of a transparent material and can cover the slice area to reflect the geometric distortion displacement of each pixel in the sliced ​​image.

[0079] Preferably, the color calibration area consists of three monochrome color bars: red, green, and blue. Each color bar includes monochrome colors with different depths, and the color depth gradually increases from left to right.

[0080] Preferably, the true color of the color bar of the multi-resolution calibration template is obtained by measuring with a colorimeter.

[0081] Preferably, the multi-resolution translation calibration area is a combination of crosshairs, and the intersection of each crosshair is the position for a translation measurement.

[0082] Preferably, the multi-resolution rotation calibration region is a radial line, which indicates the rotation direction of the slice.

[0083] Preferably, the line segment widths of the multi-resolution geometric distortion calibration region, the multi-resolution translation calibration region, and the multi-resolution rotation calibration region are of multiple resolutions (0.25 micrometers to 2 micrometers), corresponding to the single pixel size of images magnified by 40x, 20x, 10x, and 5x respectively; line segments of 0.25-2 micrometers are imaged as single-pixel wide line segments on the corresponding 40x-5x images; the center lines of the multi-resolution line segments coincide, such as... Figure 8 and 9 As shown.

[0084] In practical use, the following steps are included:

[0085] Step 01: Place the pathological slide in the slicing area, and use a high-resolution imaging system and a low-resolution imaging system to image the multi-resolution calibration template to obtain high / low-resolution images of the pathological slide containing calibration information: and

[0086] Step 02: Subpixel-level geometric distortion detection of high / low resolution images

[0087] Based on the deformation of the mesh due to geometric distortion, obtain or pixel p at the network intersection xy The deformation (dx, dy) LR , (dx, dy) HR Specific image distortion reference Figure 6 As shown, the details of the multi-resolution grid marker information are as follows: Figure 8 As shown.

[0088] The specific steps are as follows: process them separately. or image.

[0089] S021: Extract the coordinates of the template geometric center. The red horizontal line and blue vertical line at the center of the bottom row of the multi-resolution translation calibration area, which is one pixel wide for edge detection, represent the center of the template geometry. Figure 5 and Figure 9 The equations of vertical and horizontal lines are detected using the Hough transform, and the coordinates of their intersection point are analyzed and used as the coordinates of the geometric center of the template.

[0090] S022: Edge detection of single-pixel-wide geometric distortion regions along red horizontal and green vertical lines, obtaining arbitrary pixel points p at the grid points where the lines intersect. xy Pixel coordinates, such as Figure 6 Subtract the coordinates of the template's geometric center to obtain its coordinates relative to the template's geometric center.

[0091] S023: Based on the known calibration region grid points p xy Subtract the coordinates of the template geometric center from the position of the grid point to obtain the coordinates of the undistorted grid points relative to the template geometric center.

[0092] S024: Subtract the coordinates obtained from S022 and S023 to obtain the pixel p. xy The amount of deformation.

[0093] S025: Repeat S022, S023, and S024 until the deformation of all pixels at the grid intersection is obtained.

[0094] S026: The deformation of pixels at non-mesh intersections is obtained by interpolation using the known deformation of pixels at mesh intersections.

[0095] Preferably, 5x low-resolution imaging At that time, 2-micron horizontal and vertical lines indicating the geometric center of the template were used to determine the coordinates of the template's geometric center. 40x high-resolution imaging. At that time, the 0.25-micron horizontal and vertical lines marking the geometric center of the template are used to determine the coordinates of the geometric center of the template, such as... Figure 5 and Figure 9 .

[0096] Preferably, edge detection is performed using monochrome yellow and green lines to extract mesh lines from the multi-resolution geometric distortion calibration region. This achieves 5x low-resolution imaging. At this time, the intersections of 2-micrometer horizontal lines (or horizontal lines in the same row) and vertical lines (or vertical lines in the same row) are used to determine the coordinates of the grid intersection points. 40x high-resolution imaging. At that time, the intersection of a 0.25-micrometer horizontal line (or a horizontal line in the same row) and a vertical line (or a vertical line in the same row) is used to determine the coordinates of the grid point intersection, such as... Figure 8 .

[0097] Preferably, the equations of the grid lines are determined using the Hough transform, and then the coordinates of the intersection points of the grid lines are determined.

[0098] Step 03: Color normalization of high / low resolution images.

[0099] Based on the color changes in the color calibration area, obtain or The color distribution and its correspondence with real colors. (Reference) Figure 7 As shown, or A color transformation lookup table is constructed using the three colors red, green, and blue. The first column of the table contains the actual color values ​​of the color bars measured by a colorimeter, and the second column in the same row contains the color values ​​of the color bars on the color calibration area image.

[0100] Step 04: Subpixel-level relative translation detection for high / low resolution images.

[0101] Find the coordinates F of the j-th crosshair center point from the multi-resolution translation calibration region of the high / low resolution image. LR，j (x, y) and F HR，j (x, y) Calculate the relative translation between two images.

[0102] The specific steps are as follows:

[0103] S041: Extraction The position of the crosshairs in the multi-resolution translation calibration area.

[0104] The red horizontal line and blue vertical line of the multi-resolution translation calibration area with a single pixel width for edge detection, such as... Figure 9 The equations of vertical and horizontal lines are determined using the Hough transform, and the coordinates of their intersection points are analyzed and used as the coordinates of the centers of each crosshair. For example, the coordinates of the j-th center point are F. LR，j (x, y).

[0105] S042: Yes The geometric center is obtained by averaging the coordinates of all center points. Where F LR，j (x, y) is The coordinates of the j-th center point of the translation calibration area, where N is the number of center points.

[0106] S043: Similar to S041, extraction processing The position of the crosshairs in the multi-resolution translation calibration area.

[0107] S044: Yes The geometric center is obtained by averaging the coordinates of all center points. Where F HR，j (x, y) is The coordinates of the j-th center point of the translation calibration area, where N is the number of center points.

[0108] S045: Calculate relative position: Where ρ is the magnification factor of the high-resolution image relative to the low-resolution image.

[0109] Preferably, 5x low-resolution imaging At that time, the 2-micrometer horizontal and vertical lines in the multi-resolution translation calibration area were used to determine the coordinates of the crosshair center. 40x high-resolution imaging. At that time, the 0.25-micron horizontal and vertical lines of the multi-resolution translation calibration area are used to determine the coordinates of the crosshair center.

[0110] Preferably, when selecting a 40x high-resolution image and a 5x low-resolution image, the magnification factor ρ is 8.

[0111] Step 05: Subpixel-level relative rotation angle detection for high / low resolution images.

[0112] The low-resolution image is translated by [Δx, Δy], and then the rotation angle θ of the radiation line is found from the multi-resolution rotation calibration region of the high / low resolution images, which is used as the relative rotation angle Δθ between the two images.

[0113] The specific steps are as follows:

[0114] S051: Extraction The position of the radiation in the multi-resolution rotating calibration region.

[0115] The edge detection uses red horizontal lines, blue vertical lines, and green 45° / 135° lines in a multi-resolution rotation calibration area with a single pixel width. Figure 5 and Figure 9 The equations of each line are examined using the Hough transform to obtain its rotation angle. For example, the rotation angle of the j-th radial line is θ. LR，j，水平 θ LR，j，垂直 θ LR，j，45 θ LR，j，135 .

[0116] S052: Extraction The position of the radiation in the multi-resolution rotating calibration region.

[0117] The edge detection uses red horizontal lines, blue vertical lines, and green 45° / 135° lines in a multi-resolution rotation calibration area with a single pixel width. Figure 5 and Figure 9 The equations of each line are examined using the Hough transform to obtain its rotation angle. For example, the rotation angle of the j-th radial line is θ. HR，j，水平 θ HR，j，垂直 θ HR，j，45 θ HR，j，135 .

[0118] S053: Calculate the relative rotation angle: N represents the number of radiation rays.

[0119] Preferably, 5x low-resolution imaging At that time, the 2-micron horizontal and vertical lines in the multi-resolution rotation calibration region were used to determine the radiation angle. 40x high-resolution imaging. At that time, the 0.25-micron horizontal and vertical lines of the multi-resolution rotation calibration area are used to determine the radiation angle.

[0120] Step 06: Treat the transformation between high / low resolution images as a nonlinear rigid body transformation. Using the translation and rotation amounts obtained from the calibration in Steps 04-05, construct the translation matrix F and rotation matrix G, and merge them into the rigid body transformation matrix A. Simultaneously, construct the nonlinear transformation matrix from the pixel deformation field obtained from the multi-resolution geometric distortion calibration region. Align the coordinates of all pixels in the low-resolution image with the coordinates of all pixels in the high-resolution image, as shown below:

[0121]

[0122]

[0123]

[0124] Step 07: Perform color correction on the pixel-aligned images individually, using a color transformation lookup table ( Figure 7 This yields high / low resolution images that match the true color values ​​of the color bars.

[0125] The specific steps are as follows: process them separately. or Use their respective color transformation lookup tables.

[0126] S071: Read or Any pixel p in the image xy The color value is RGB.

[0127] S072: Find the true R value (first column) corresponding to the R value (second column) in the color transformation lookup table; find the true G value (first column) corresponding to the G value (second column); find the true B value (first column) corresponding to the B value (second column).

[0128] S073: Obtain pixel p xy The true color value (RGB) is used to replace the original color value.

[0129] S074: Yes or Repeat steps S071-S073 for all pixels.

[0130] S075: Extract high / low resolution images of pathological tissue from the slice region of the multi-resolution calibration template, and use them as calibrated high / low resolution image pairs for training the computational model. LR and I HR .

[0131] Step 08: Replace with different pathological slides and repeat steps 01-07 to obtain a dataset of corrected high / low resolution image pairs {I}. LR I HR}

[0132] Preferably, the position of the fixed calibration area remains unchanged, and the previously calculated relative translation, rotation, geometric distortion, and color lookup table of the high / low resolution images can be reused to execute steps 06 and 07 to simplify the processing.

[0133] Step 09: Use the dataset of high / low resolution image pairs {I LR I HR A computational model is trained using a supervised end-to-end approach to implement the input low-resolution image I. LR Output the generated high-resolution image I SR .

[0134] Preferably, the computational model can be designed using super-resolution generative models such as generative adversarial networks or encoder-decoder networks.

[0135] Preferably, during training, the high / low resolution image pair dataset {I} LR I HR 60% of the data is used as the training set, 20% as the validation set, and the remaining 20% ​​as the test set. A backpropagation neural network training algorithm is employed, continuously optimizing the model weights during training to achieve a result from I... LR The generated I SR and pairs of I HR The training target has image features that are highly similar to each other.

[0136] Preferably, the training objective function limits the generated I SR and I HR They exhibit a high degree of similarity, including similarity in pixels, colors, textures, and perceptual features.

[0137] Step 10: After training the computational model, save the model to the parallel computing unit for use in low-resolution imaging processes, from I... LR Generate high-resolution image I SR .

[0138] Step 11: When performing low-resolution pathological slide imaging, use a low-resolution imaging system to image the pathological slides and obtain... Low-resolution image I is obtained after image correction. LR .

[0139] Preferably, since pixel alignment of the high-resolution image is not required, the correction here only needs to perform geometric distortion correction, and there is no need to perform selective translation and rotation correction, as shown below.

[0140]

[0141] Preferably, further color correction is performed. The colors are standardized.

[0142] Preferably, the pixel deformation amount and color correction lookup table required for geometric distortion correction and color correction are obtained by steps 02 and 03.

[0143] Preferably, during low-resolution imaging, each calibration region of the multi-resolution calibration template is removed, and imaging is performed only on pathological sections to simplify the imaging system.

[0144] Step 12: The parallel computing unit runs the trained computational model, realizing the computation by I LR (Low-resolution image) generation I SR (High-resolution image)

[0145] Preferably, after the calibration system completes the construction of the computational model, it is abandoned when performing low-resolution imaging of pathological slides to save costs.

[0146] Preferably, during low-resolution imaging, a low-resolution image I is output simultaneously. LR and high-resolution image I SR To meet the needs of multi-resolution pathological diagnostic applications using high / low resolution images.

[0147] Preferably, the parallel computing unit can be a high-performance computing module such as a GPU, which includes thousands of computing cores and can achieve parallel high-speed operation.

[0148] Preferably, each low-resolution imaging system can be equipped with multiple parallel processing units to achieve a balance between parallel computing speed and optical scanning imaging speed, thereby maximizing the imaging speed.

[0149] Preferably, the parallel computing units and computing models can be moved to a parallel server system so that the parallel server system can serve multiple low-resolution imaging systems simultaneously, thereby reducing the economic cost of the low-resolution imaging systems.

[0150] As can be seen from the above description, in this embodiment, by employing a multi-resolution calibration template, high / low resolution image pairs with corrected color and geometric distortion, and precise pixel alignment, are obtained, achieving an advancement in the accurate training of the computational model. Through low-resolution imaging and computational model conversion, a new digital pathological slide technique based on low-resolution imaging has been established, solving the problems of slow speed, local defocusing, and geometric distortion in current high-resolution imaging digital slides, saving digitization time, and improving image quality.

[0151] In addition, this application embodiment also provides a low-resolution imaging digitization system for pathological slides. This system includes a low-resolution imaging system comprising: a low-magnification lens, an optoelectronic device, a signal processing unit, an image correction unit, and a parallel computing unit. The low-magnification lens is used to scan pathological slides to obtain a corresponding low-magnification optically magnified image. The optoelectronic device converts the optically magnified image into an analog electrical signal. The signal processing unit performs analog-to-digital conversion on the analog electrical signal to obtain a low-resolution image. The image correction unit processes image color and geometric distortion. The parallel computing unit converts the low-resolution image into a high-resolution image based on an established computational model that can reconstruct the low-resolution image of pathological tissue into a high-resolution image. For details, please refer to the detailed description in the above method; further elaboration is omitted here.

[0152] In the above technical solution, by employing a multi-resolution calibration template, high / low resolution image pairs with corrected color and geometric distortion, and precise pixel alignment, were obtained, achieving an advancement in accurate training of the computational model. Through low-resolution imaging and computational model conversion, a new digital pathological slide technique based on low-resolution imaging was established, solving the problems of slow speed, local defocusing, and geometric distortion in current high-resolution imaging digital slides, saving digitization time and improving image quality.

[0153] One or more embodiments of this specification are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the scope of protection of this disclosure.

[0154] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of low resolution imaging of a pathology slide digitization, characterized in that, The method comprises the following steps: imaging the multi-resolution calibration template by using a low-resolution imaging system and a high-resolution imaging system respectively; obtaining low-resolution images and high-resolution images of pathological tissue, color, geometric shape and positioning information of the multi-resolution calibration template; preprocessing and image correcting the low-resolution images and high-resolution images of the pathological tissue according to the color, geometric shape and positioning information of the multi-resolution calibration template, to obtain a pair of high / low-resolution images of the pathological tissue which are color-standardized, geometric-distortion-corrected and pixel-aligned; training a super-resolution image reconstruction algorithm in a supervised manner by using a plurality of pairs of high / low-resolution images, to establish a calculation model which can reconstruct a high-resolution image from a low-resolution image of the pathological tissue; imaging the pathological section by using the low-resolution imaging system, and converting the low-resolution image into a high-resolution image by using the calculation model; the multi-resolution calibration template comprises a color calibration area, a multi-resolution geometric distortion calibration area, a multi-resolution translation calibration area, a multi-resolution rotation calibration area and a section area; wherein: the color calibration area and the multi-resolution geometric distortion calibration area are used to provide color distribution and geometric distortion of the high-resolution imaging system and the low-resolution imaging system; the multi-resolution translation calibration area and the multi-resolution rotation calibration area provide relative translation and rotation of the high / low-resolution images; and the section area is used to place different pathological sections.

2. The method of low resolution imaging of pathology slides digitization according to claim 1, characterized in that, the imaging of the multi-resolution calibration template by using the low-resolution imaging system and the high-resolution imaging system respectively comprises: the low-resolution imaging system scans a certain number of image blocks of the multi-resolution calibration template to obtain a low-resolution image of the multi-resolution calibration template by splicing; the high-resolution imaging system scans a certain number of image blocks of the multi-resolution calibration template to obtain a high-resolution image of the multi-resolution calibration template by splicing.

3. The method of low resolution imaging of pathology slides digitization according to claim 2, characterized in that, The geometric distortion, translation and rotation mark information on the multi-resolution calibration template image are multi-resolution line segments, a specific resolution line segment presents a single-pixel width in the high / low-resolution image, and the center lines of the multi-resolution line segments coincide, to obtain sub-pixel level positioning accuracy, wherein the multi-resolution calibration template image refers to an image obtained after imaging the multi-resolution calibration template.

4. The method of low resolution imaging of pathology slides digitization of claim 2, wherein, The color calibration area on the multi-resolution calibration template image is used to provide color distribution of the imaging system, and specifically comprises: the color distribution and the corresponding relationship with the true color of the low-resolution image and the high-resolution image are obtained through the color change of the color calibration area, to realize color standardization; wherein the multi-resolution calibration template image refers to an image obtained after imaging the multi-resolution calibration template.

5. The method of low resolution imaging of pathology slides digitization of claim 3, wherein, a pair of high / low-resolution images of the pathological tissue which are geometric-distortion-corrected and pixel-aligned are obtained; specifically comprising: the deformation amount of the pixels of the high-resolution image and the low-resolution image is obtained through the multi-resolution geometric distortion calibration area; the relative rotation angle of the pixels of the high-resolution image and the low-resolution image is obtained through the multi-resolution rotation calibration area; The relative rotation angle of pixels of the high-resolution image and the low-resolution image is obtained through the multi-resolution rotation calibration area.

6. The method of low resolution imaging of pathology slides digitization according to claim 5, characterized in that, The relative rotation angle of pixels of the high-resolution image and the low-resolution image is obtained through the multi-resolution rotation calibration area; the relative translation amount of pixels of the high-resolution image and the low-resolution image is obtained through the multi-resolution translation calibration area; specifically: The deformation amount of pixels of the high-resolution image and the low-resolution image is obtained through the multi-resolution specific mark of the multi-resolution geometric distortion calibration area; the translation amount and the rotation amount of pixels of the high-resolution image and the low-resolution image are obtained through the multi-resolution specific mark of the multi-resolution translation calibration area and the multi-resolution rotation calibration area, a non-linear rigid transformation matrix is constructed, and the pixel position alignment of the high-resolution image and the low-resolution image is realized; Through the color normalization, geometric distortion correction and pixel alignment, a high / low resolution image pair is established.

7. The method of low resolution imaging of pathology slides digitization according to claim 6, characterized in that, The calculation model of the super-resolution image reconstruction algorithm is trained in a supervised manner by using a plurality of high / low resolution image pairs, and specifically includes: The training target function is to make the high-resolution image generated from the low-resolution image and the paired high-resolution image have high similarity, and the high similarity includes the similarity of pixels, colors, textures and perceptual features.

8. A low resolution imaging of pathological sections digitization system characterized in that, The low-resolution imaging system includes a low magnification lens, a photoelectric device, a signal processing unit, an image correction unit and a parallel computing unit. The low magnification lens is used to scan the pathological section to obtain a corresponding low magnification optical magnification image; the photoelectric device is used to convert the optical magnification image into an analog electrical signal; the signal processing unit is used to perform analog-to-digital conversion on the analog electrical signal to obtain a low-resolution image; the image correction unit is used to correct the image color and pixel deformation; and the parallel computing unit is used to convert the low-resolution image into a high-resolution image according to the established calculation model which can reconstruct the low-resolution image of the pathological tissue into a high-resolution image. The calculation model which can reconstruct the low-resolution image of the pathological tissue into a high-resolution image is established by training the super-resolution image reconstruction algorithm in a supervised manner by using a plurality of high / low resolution image pairs. The high / low resolution image pair is obtained by pre-processing and image correction of the low-resolution image and the high-resolution image of the pathological tissue according to the color, geometric shape and positioning information of the multi-resolution calibration template, and the high / low resolution image pair of the pathological tissue with color normalization, geometric distortion correction and pixel alignment is obtained. The multi-resolution calibration template includes a color calibration area, a multi-resolution geometric distortion calibration area, a multi-resolution translation calibration area, a multi-resolution rotation calibration area and a section area; wherein: The color calibration area and the multi-resolution geometric distortion calibration area are used to provide the color distribution and the geometric distortion amount of the high-resolution imaging system and the low-resolution imaging system; the multi-resolution translation calibration area and the multi-resolution rotation calibration area provide the relative translation and rotation amount of the high / low resolution image; and the section area is used to place different pathological sections.

Images