Microscope-based digital slide generation system, method and related equipment

Abstract

Embodiments of the present application disclose a digital slide generation system, method, and related device based on a microscope. By acquiring a digital slide image corresponding to slide reading and coordinate information corresponding to the digital slide image while using the microscope for slide reading, and fusing the digital slide images based on the coordinate information, it is possible to accurately fuse the digital slide images obtained during slide reading into a microscopic image digital slide corresponding to the object to be photographed according to the coordinate information, without separately using a digital slide scanner to perform a digital scanning operation on the object to be photographed, thereby simplifying the workflow of digital slide generation, reducing the diagnosis time, and reducing the diagnosis costs of pathology or other disciplines using microscopes.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and particularly to a digital slide generation system, method and related device based on a microscope. Background Art

[0002] Pathological diagnosis is a process of identifying diseases by analyzing slices of human tissues and cells. A microscope is an indispensable tool in pathological diagnosis. Pathologists use an ordinary optical microscope to examine the microscopic structure of tissue samples taken from a diseased site through staining techniques (such as H&E staining), including cell morphology, arrangement, and tissue structure, so as to determine whether there are abnormalities. The optical microscope is also used for slice examination, identifying the distribution and expression of specific types of proteins in tissues through immunohistochemical staining, and so on. In addition to optical microscopes, fluorescence microscopes can also detect infectious pathogens or specific gene expressions by labeling specific molecules or structures with fluorescent dyes. In addition to clinical pathological diagnosis, microscopes are also widely used in disciplines such as microbiology, bone marrow pictures, and neuroimmunology. However, existing microscopes only have the functions of image magnification and image taking, and do not have the function of slice scanning. In order to obtain digital slides, doctors need to perform separate digital scanning operations.

[0003] Currently, the mainstream tool for realizing digital slides is a "digital slide scanner". Its advantage is high image clarity, but the cost is high. Low-throughput scanners cannot scan a large number of slices, and the cost of high-throughput scanners is still very high. In addition, adding a separate scanning link will increase a workflow, extend the diagnosis time, and increase the diagnosis cost of pathology or other disciplines using microscopes. Summary of the Invention

[0004] In view of this, the present invention provides a digital slide generation system, method and related device based on a microscope.

[0005] The specific technical solution of the first embodiment of the present invention is: A digital slide generation system based on a microscope, the system includes: a microscope, a shooting module, a positioning module, and a central control module; the shooting module is connected to the central control module, and the output end of the positioning module is connected to the input end of the central control module;

[0006] When the object to be photographed is viewed through the microscope at different positions, the central control module is used to send a photographing instruction to the photographing module and send a positioning instruction to the positioning module; the photographing module is used to photograph digital section images of the object to be photographed at different positions after being magnified by the microscope according to the photographing instruction, and send the digital section images at different positions to the central control module; the positioning module is used to obtain coordinate information of different positions when the object to be photographed is photographed by the photographing module according to the positioning instruction, and send the coordinate information of different positions to the central control module; the central control module is used to perform image fusion on the digital section images at different positions and the coordinate information of different positions to obtain a microscopic image digital section of the object to be photographed.

[0007] Preferably, the positioning module obtains the Y-axis coordinate information in the coordinate information by detecting the distance between a preset fixed point of the stage moving platform of the microscope and the microscope wall; and obtains the X-axis coordinate information in the coordinate information by detecting the distance between the preset fixed point and the stage section clamp of the microscope.

[0008] The specific technical solution of the second embodiment of the present invention is: a method for generating digital sections based on a microscope, which is applied to the digital section generation system based on a microscope as described in the first embodiment of the present application. The method includes: when the object to be photographed is viewed through the microscope at different positions, sending a photographing instruction to the photographing module and sending a positioning instruction to the positioning module; controlling the photographing module to photograph digital section images of the object to be photographed at different positions after being magnified by the microscope; controlling the positioning module to obtain coordinate information of different positions when the object to be photographed is photographed by the photographing module; and performing image fusion on the digital section images at different positions and the coordinate information of different positions to obtain a microscopic image digital section of the object to be photographed.

[0009] Preferably, the microscope includes objective lenses with different magnification multiples. After controlling the photographing module to photograph digital section images of the object to be photographed at different positions after being magnified by the microscope, it further includes: obtaining the magnification multiples of the objective lenses corresponding to the digital section images at different positions; and performing image fusion on the digital section images at different positions according to the magnification multiples of the objective lenses and the coordinate information of different positions to obtain a microscopic image digital section of the object to be photographed.

[0010] Preferably, obtaining the microscopic image digital slice of the object to be photographed by fusing the digital slice images at different positions according to the magnification of the objective lens and the coordinate information at different positions includes: constructing a movement curve of the object to be photographed according to the coordinate information at different positions; scaling the digital slice images according to the magnification of the objective lens corresponding to the digital slice images at different positions, wherein all the scaled digital slice images have the same magnification; and fusing all the scaled digital slice images according to the coordinate information on the movement curve to obtain the microscopic image digital slice.

[0011] Preferably, the magnification of the objective lens includes a first magnification, a second magnification, and a third magnification; the first magnification is less than the second magnification, and the second magnification is less than the third magnification; then, the method for obtaining the microscopic image digital slice of the object to be photographed by performing image fusion on the digital slice images at different positions according to the magnification of the objective lens and the coordinate information at different positions includes: constructing a first layer corresponding to the first magnification, constructing a second layer corresponding to the second magnification, and constructing a third layer corresponding to the third magnification; saving the first target digital slice image obtained by using the first magnification into the first layer, saving the second target digital slice image obtained by using the second magnification into the second layer, and saving the third target digital slice image obtained by using the third magnification into the third layer; performing a puzzle operation on all the first target digital slice images in the first layer according to the coordinate information of the first target digital slice image to obtain a first slice image; performing a puzzle operation on all the second target digital slice images in the second layer according to the coordinate information of the second target digital slice image to obtain a second slice image; performing a puzzle operation on all the third target digital slice images in the third layer according to the coordinate information of the third target digital slice to obtain a third slice image; magnifying the first slice image according to the ratio of the second magnification to the first magnification to obtain a first simulated slice image; retrieving a first target area where the object to be photographed has no collected image in the second slice image; obtaining first image information corresponding to the first target area in the first simulated slice image and filling the first image information in the second slice image to obtain a second simulated slice image; magnifying the second simulated slice image according to the ratio of the third magnification to the second magnification to obtain a third simulated slice image; retrieving a second target area where the object to be photographed has no collected image in the third slice image; obtaining second image information corresponding to the second target area in the third simulated slice image and filling the second image information in the third slice image to obtain the microscopic image digital slice.

[0012] Preferably, the method further includes: when performing image fusion on the digital slice images at different positions, if there is an overlapping image area between the first digital slice image and the second digital slice image, obtaining the first picture clarity of the image overlapping area corresponding to the first digital slice image, and obtaining the second picture clarity of the image overlapping area corresponding to the second digital slice image; the first digital slice image and the second digital slice image are any one of all the digital slice images; comparing the first picture clarity and the second picture clarity; if the first picture clarity is greater than the second picture clarity, selecting the fourth target digital slice image of the first digital slice image in the image overlapping area, and using the fourth target digital slice image as the image of the image overlapping area after image fusion; if the first picture clarity is less than the second picture clarity, selecting the fifth target digital slice image of the second digital slice image in the image overlapping area, and using the fifth target digital slice image as the image of the image overlapping area after image fusion.

[0013] Preferably, the obtaining of the digital slice images at different positions of the object to be photographed after being magnified by the microscope includes: obtaining the first field of view content of the object to be photographed after being magnified by the microscope, and photographing the first field of view content to obtain a digital slice image corresponding to the first field of view content; moving the object to be photographed so that the first field of view content changes to the second field of view content; there is a preset percentage of the field of view content that is different between the second field of view content and the first field of view content; photographing the second field of view content to obtain a digital slice image corresponding to the second field of view content, and using the second field of view content as the first field of view content, and returning to the step of moving the object to be photographed so that the first field of view content changes to the second field of view content until all the digital slice images corresponding to all the field of view contents are obtained; all the digital slice images corresponding to all the field of view contents are the digital slice images at different positions.

[0014] The specific technical solution of the third embodiment of the present invention is: a digital slice generation device based on a microscope, including a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the method according to any one of the second embodiments of the present application.

[0015] The specific technical solution of the fourth embodiment of the present invention is: a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the steps of the method according to any one of the second embodiments of the present application.

[0016] Implementing the embodiments of the present invention will have the following beneficial effects:

[0017] By acquiring the digital slice image corresponding to the film reading and the coordinate information corresponding to the digital slice image while using a microscope for film reading, and fusing the digital slice image based on the coordinate information, the digital slice images obtained during film reading can be accurately fused into a microscopic image digital slice corresponding to the object to be photographed according to the coordinate information, without separately using a digital slice scanner to perform a digital scanning operation on the object to be photographed, thereby simplifying the workflow of digital slice generation, reducing the diagnosis time, and reducing the diagnosis cost of pathology or other disciplines using microscopes. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0019] Figure 1 is a schematic structural diagram of a digital slice generation system based on a microscope;

[0020] Figure 2 is a flowchart of the steps of the first embodiment of a digital slice generation method based on a microscope;

[0021] Figure 3 is a flowchart of the steps of image fusion based on coordinate information;

[0022] Figure 4 is a flowchart of the steps of the second embodiment of a digital slice generation method based on a microscope;

[0023] Figure 5 is an internal structural diagram of a computer device;

[0024] Among them, 101, microscope; 102, shooting module; 103, positioning module; 104, central control module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] The following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0026] The terms "first", "second", etc. in the description, claims and drawings of this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but optionally also includes steps or modules not listed, or optionally also includes other steps or modules inherent to these processes, methods, products or devices.

[0027] Reference to "embodiment" herein means that a particular feature, structure or characteristic described in connection with the embodiment can be included in at least one embodiment of this application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0028] Please refer to Figure 1 , which is a schematic structural diagram of a digital slide generation system based on a microscope in the first embodiment of this application. To simplify the workflow of digital slide generation and reduce the diagnosis time, the system includes: a microscope 101, a shooting module 102, a positioning module 103 and a central control module 104; the shooting module 102 is connected to the central control module 104, and the output end of the positioning module 103 is connected to the input end of the central control module 104; when the object to be photographed is read at different positions through the microscope, the central control module is used to send a shooting instruction to the shooting module and send a positioning instruction to the positioning module; the shooting module is used to shoot digital slide images of the object to be photographed at different positions after being magnified by the microscope according to the shooting instruction, and send the digital slide images at different positions to the central control module; the positioning module is used to obtain the coordinate information of different positions when the object to be photographed is photographed by the shooting module according to the positioning instruction, and send the coordinate information of different positions to the central control module; the central control module is used to perform image fusion on the digital slide images at different positions and the coordinate information at different positions to obtain a microscopic image digital slide of the object to be photographed.

[0029] Specifically, the shooting module can be an electronic objective lens, and the positioning module can be a ranging component such as a laser distance sensor or an optical scale. The positioning module identifies the current coordinate information of the microscope stage and transmits it to the central control module in real time. Among them, the optical scale sensor is integrated into the X / Y axis guide rail of the stage and synchronizes with the moving slider. The laser sensor is an external type, fixed to the stage base and pointing to the reflecting surface of the slide clamp to ensure that it is not affected by vibration during use.

[0030] When the object to be photographed is viewed through a microscope at different positions, the central control module sends a photographing instruction to the photographing module and a positioning instruction to the positioning module; the photographing module photographs digital section images of the object to be photographed at different positions after being magnified by the microscope. For example, when the magnification of the current microscope is 40X, the photographing module will photograph digital section images of the object to be photographed magnified 40X at different positions, and send the digital section images of the object to be photographed magnified 40X to the central control module; the positioning module obtains coordinate information at different positions according to the positioning instruction and sends the coordinate information to the central control module; the central control module performs image fusion on the digital section images at different positions and the coordinate information at different positions to obtain a microscopic image digital section of the object to be photographed.

[0031] In the system of this embodiment, by obtaining the digital section image corresponding to the film reading and the coordinate information corresponding to the digital section image while using the microscope for film reading, and performing fusion on the digital section image based on the coordinate information, it is possible to accurately fuse the digital section images obtained during film reading into the microscopic image digital section corresponding to the object to be photographed according to the coordinate information, without separately using a digital section scanner to perform digital scanning operations on the object to be photographed, thereby simplifying the workflow of digital section generation, reducing the diagnosis time, and reducing the diagnosis cost of pathology or other disciplines using microscopes.

[0032] In a specific embodiment, the positioning module obtains the Y-axis coordinate information in the coordinate information by detecting the distance between a preset fixed point on the moving platform of the microscope stage and the microscope wall; and obtains the X-axis coordinate information in the coordinate information by detecting the distance between the preset fixed point and the slide clamp on the microscope stage.

[0033] Specifically, when the positioning module is a laser distance sensor, the laser distance sensor is set at the upper left corner of the moving platform of the stage. The distance between the laser distance sensor and the microscope wall is the Y-axis coordinate information, and the distance between the laser distance sensor and the slide clamp on the stage is the X-axis coordinate information. A reflective point can also be set in the direction where the laser distance sensor measures the Y-axis coordinate information, relative to the position of the laser distance sensor. The distance between the laser distance sensor and the reflective point is the Y-axis coordinate information.

[0034] In a specific embodiment, please refer to Figure 2 , the flowchart of the steps of a method for generating digital sections based on a microscope provided in the second embodiment of the present application is applied to the system for generating digital sections based on a microscope as described in the first embodiment of the present application. The method includes:

[0035] Step 201, when the object to be photographed is viewed through the microscope at different positions, send a photographing instruction to the photographing module and a positioning instruction to the positioning module;

[0036] Step 202: Control the shooting module to shoot digital slice images of the object to be shot at different positions after being magnified by the microscope.

[0037] Step 203: Control the positioning module to obtain coordinate information of different positions when the object to be shot is shot by the shooting module.

[0038] Step 204: Perform image fusion on the digital slice images at different positions and the coordinate information at different positions to obtain a digital slice of the microscopic image of the object to be shot.

[0039] Specifically, when the object to be shot is viewed at different positions through the microscope, a shooting instruction is sent to the shooting module and a positioning instruction is sent to the positioning module; control the shooting module to shoot digital slice images of the object to be shot at different positions after being magnified by the microscope. For example, when the magnification of the current microscope is 40X, the shooting module will shoot digital slice images of the object to be shot magnified 40X at different positions, control the positioning module to obtain coordinate information of different positions, and perform image fusion on the digital slice images at different positions and the coordinate information at different positions to obtain a digital slice of the microscopic image of the object to be shot, so as to accurately fuse the digital slice images obtained during viewing according to the coordinate information into a digital slice of the microscopic image corresponding to the object to be shot, without separately using a digital slice scanner to perform digital scanning operations on the object to be shot, thereby simplifying the workflow of generating digital slices, reducing the diagnosis time, and reducing the diagnosis cost of pathology or other disciplines using microscopes.

[0040] In a specific embodiment, the microscope includes objective lenses with different magnifications. After controlling the shooting module to shoot digital slice images of the object to be shot at different positions after being magnified by the microscope, it further includes: obtaining the magnification of the objective lens corresponding to the digital slice images at different positions; and performing image fusion on the digital slice images at different positions and the coordinate information at different positions to obtain a digital slice of the microscopic image of the object to be shot, including: performing image fusion on the digital slice images at different positions according to the magnification of the objective lens and the coordinate information at different positions to obtain a digital slice of the microscopic image of the object to be shot.

[0041] Specifically, objective lenses with different magnification factors are set in the microscope, such as a low-power lens of 40X, a medium-power lens of 100X, or a high-power lens of 200X. When the user uses the microscope to view slides, different magnification objective lenses will be used according to the needs. For example, when viewing the A area of the object to be photographed, a 40X low-power lens is used for viewing. When viewing the B area of the object to be photographed, a 100X medium-power lens is used for viewing. Therefore, the objective lens magnification factors of the obtained digital slide images will be inconsistent. For this reason, the objective lens magnification factor corresponding to each digital slide image will be obtained first, and then according to the objective lens magnification factor and the coordinate information of different positions, the digital slide images at different positions will be image-fused to obtain the microscopic image digital slide of the object to be photographed, so as to achieve the accurate fusion of digital slide images with different objective lens magnification factors.

[0042] In a specific embodiment, the objective lens magnification factor is obtained in the following manner:

[0043] 1. Objective lens switching sensor: A Hall sensor is used. When a magnet is embedded in the objective lens turntable, the Hall signal is triggered when the objective lens is in place. The central control module collects the Hall signal, and different Hall signals represent different objective lens magnification factors; or an optoelectronic sensor is used to detect the notch position of the objective lens turntable, and different notch positions represent different objective lens magnification factors.

[0044] 2. Magnification coding mapping: The objective lenses with different magnifications are coded. For example, the 40X objective lens corresponds to the code 01, the 100X objective lens corresponds to the code 10, and the 200X objective lens corresponds to the code 11. The central processing module identifies the corresponding code to obtain the objective lens magnification factor.

[0045] In a specific embodiment, please refer to Figure 3 , the image-fusing the digital slide images at different positions according to the objective lens magnification factor and the coordinate information of different positions to obtain the microscopic image digital slide of the object to be photographed includes:

[0046] Step 301: Construct a movement curve of the object to be photographed according to the coordinate information of different positions;

[0047] Step 302: Image-scale the digital slide images according to the objective lens magnification factor corresponding to the digital slide images at different positions; wherein, all the scaled digital slide images have the same magnification factor;

[0048] Step 303: On the movement curve, fuse all the scaled digital slide images according to the coordinate information to obtain the microscopic image digital slide.

[0049] Specifically, the function of the moving curve is to initially place a large number of digital slice images on the jigsaw blackboard, facilitating the selection of digital slice images above, below, left, and right during subsequent calculations. Additionally, when there is no overlap between digital slice images, discontinuous digital slice images can be placed in a roughly correct position. After placing the digital slice images at the center points on the moving curve, each image takes the center point coordinates as the center of the circle, and the center point coordinate images within a retrieval radius of 1 times the major axis are marked as adjacent images. First, digital slice images with different magnifications are scaled to digital slice images with a unified magnification, such as scaling to images of 40X or 100X, etc. For images with the same magnification, starting from the actual moving coordinates of the moving curve, the relative displacement amounts between the adjacent images and the current image of each image are calculated respectively, and the relative displacement amounts are adjusted more precisely by calculating the overlapping areas, and microscopic stitching is performed at the cellular level.

[0050] In a specific embodiment, the method further includes: when performing image fusion on digital slice images at different positions, if there is an image area overlap between a first digital slice image and a second digital slice image, obtaining the first picture clarity of the image overlap area corresponding to the first digital slice image, and obtaining the second picture clarity of the image overlap area corresponding to the second digital slice image; the first digital slice image and the second digital slice image are any digital slice image among all the digital slice images; comparing the first picture clarity and the second picture clarity; if the first picture clarity is greater than the second picture clarity, selecting a fourth target digital slice image of the first digital slice image in the image overlap area, and using the fourth target digital slice image as the image of the image overlap area after image fusion; if the first picture clarity is less than the second picture clarity, selecting a fifth target digital slice image of the second digital slice image in the image overlap area, and using the fifth target digital slice image as the image of the image overlap area after image fusion.

[0051] In a specific embodiment, for the overlapping area, a preset software algorithm is used to read the clarity of different layers in the overlapping area and perform numerical quantization. Select the image with a higher clarity value as the display image of the overlapping area, and delete the remaining images to improve the running speed of image fusion. The entire image is scanned alternately to finally achieve clear scanning of most areas.

[0052] Specifically, the clarity of an image is measured by calculating the edge intensity (i.e., clarity) of the image. The Sobel operator and gradient calculation are used to detect the edges of the image, and the clarity of the image is quantified by calculating the intensity of the edges.

[0053] Calculate the gradient using the Sobel operator: gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY); sobel_x = cv2.Sobel(gray, cv2.CV_64F, 1, 0, ksize=3); sobel_y = cv2.Sobel(gray, cv2.CV_64F, 0, 1, ksize=3).

[0054] If the mean of the gradient magnitude is used as the sharpness metric: magnitude = np.sqrt(sobel_x2 + sobel_y2); return np.mean(magnitude). If the edge intensity of the image is large, it indicates that the image is sharp and details are clearly visible. If the edges of the image are blurred, it indicates that the image is of low sharpness and is blurred.

[0055] In a specific embodiment, the magnification of the objective lens includes a first magnification, a second magnification, and a third magnification; the first magnification is less than the second magnification, and the second magnification is less than the third magnification; then the method for obtaining the microscopic image digital slice of the object to be photographed by performing image fusion on the digital slice images at different positions according to the magnification of the objective lens and the coordinate information at different positions includes: constructing a first layer corresponding to the first magnification, constructing a second layer corresponding to the second magnification, and constructing a third layer corresponding to the third magnification; saving the first target digital slice image obtained by using the first magnification into the first layer, saving the second target digital slice image obtained by using the second magnification into the second layer, and saving the third target digital slice image obtained by using the third magnification into the third layer; performing a puzzle operation on all the first target digital slice images in the first layer according to the coordinate information of the first target digital slice image to obtain a first slice image; performing a puzzle operation on all the second target digital slice images in the second layer according to the coordinate information of the second target digital slice image to obtain a second slice image; performing a puzzle operation on all the third target digital slice images in the third layer according to the coordinate information of the third target digital slice to obtain a third slice image; magnifying the first slice image according to the ratio of the second magnification to the first magnification to obtain a first simulated slice image; retrieving a first target area where the object to be photographed has no collected image in the second slice image; obtaining first image information corresponding to the first target area in the first simulated slice image and filling the first image information in the second slice image to obtain a second simulated slice image; magnifying the second simulated slice image according to the ratio of the third magnification to the second magnification to obtain a third simulated slice image; retrieving a second target area where the object to be photographed has no collected image in the third slice image; obtaining second image information corresponding to the second target area in the third simulated slice image and filling the second image information in the third slice image to obtain the microscopic image digital slice.

[0056] Specifically, the objective lens magnification factors include 40X, 100X, and 200X. A first layer corresponding to 40X, a second layer corresponding to 100X, and a third layer corresponding to 200X are constructed respectively. The first target digital slice image obtained using 40X is saved into the first layer, the second target digital slice image obtained using 100X is saved into the second layer, and the third target digital slice image obtained using 200X is saved into the third layer. According to the coordinate information of the first target digital slice images in the first layer, all the first target digital slice images in the first layer are subjected to a mosaic operation to obtain a first slice image. According to the coordinate information of the second target digital slice images in the second layer, all the second target digital slice images in the second layer are subjected to a mosaic operation to obtain a second slice image. According to the coordinate information of the third target digital slices in the third layer, all the third target digital slice images in the third layer are subjected to a mosaic operation to obtain a third slice image. The first slice image magnified by 40X is magnified by 2.5 times to generate a first simulated slice image simulating 100X. The uncollected field-of-view area is retrieved in the second slice image, and this area is filled with the first simulated slice image simulating 100X to generate a second simulated slice image of 100X. The second simulated slice image of 100X is magnified by 2 times to generate a third simulated slice image simulating 200X. The uncollected field-of-view area is retrieved in the third slice image magnified by 200X, and this area is filled with the third simulated slice image simulating 200X. The image after the two filling operations is stored to generate a general digital slice image in JPG format.

[0057] In a specific embodiment, the obtaining of digital slice images at different positions of the object to be photographed after being magnified by the microscope includes: obtaining the first field-of-view content of the object to be photographed after being magnified by the microscope, and photographing the first field-of-view content to obtain a digital slice image corresponding to the first field-of-view content; moving the object to be photographed so that the first field-of-view content changes to a second field-of-view content; there is a preset percentage of the field-of-view content that is different between the second field-of-view content and the first field-of-view content; photographing the second field-of-view content to obtain a digital slice image corresponding to the second field-of-view content, and taking the second field-of-view content as the first field-of-view content, and returning to the step of moving the object to be photographed so that the first field-of-view content changes to the second field-of-view content until all digital slice images corresponding to all field-of-view contents are obtained; all the digital slice images corresponding to all field-of-view contents are the digital slice images at different positions.

[0058] Specifically, according to different magnification factors, preset values at three magnification factors of 200X (high magnification), 100X (medium magnification), and 40X (low magnification) are set respectively. The setting standard is that the one-way moving distance accounts for 50% of the current field of view width. If the movement in any direction of the X-axis or Y-axis exceeds the preset value, it is determined as an effective movement. The formula for calculating the field of view width is: Field of view width = Sensor field of view size of the objective lens / Magnification factor of the objective lens. For example, if the size of the CMOS sensor of the objective lens is 22mm, the field of view width of the 200X objective lens = 22mm / 200 = 0.11mm; the movement trigger threshold for 200X = 0.055mm (0.11mm × 50%); the movement trigger threshold for 100X = 0.11mm; the movement trigger threshold for 40X = 0.275mm; after exceeding the movement trigger threshold, the electronic objective lens is triggered to capture a real-time microscopic image of the field of view under the front lens. Trigger condition:

[0059]

[0060] Among them, is the X coordinate value of the current coordinate point, is the Y coordinate value of the current coordinate point, is the X coordinate value of the coordinate point collected last time, is the Y coordinate value of the coordinate point collected last time. After an effective movement, the electronic objective lens is triggered to take a photo. The reason for not using continuous photography is that continuous photography by the high-speed camera will generate a large amount of information, which will cause the system to run stuck.

[0061] In a specific embodiment, the complete step schematic diagram of the present application is as shown in Figure 4 During the process of digital slicing, first, the coordinate position information is obtained, and the magnification information of the microscope objective lens is collected. At different magnification factors, the coordinate movement distance is calculated. When the movement distance exceeds the preset value (50% of the field of view width at the current magnification factor), the central control module issues an instruction to trigger the camera module to take a photo. Combining the recognized absolute coordinate displacement amount, the slice movement direction information and movement distance are calculated. At the same time, combined with the AI algorithm, the more accurate movement direction and movement distance of the image under the microscope are accurately identified, and the two pieces of information are double-pieced together. For the slices of the current day, they can be set for local storage. For the slices that have been diagnosed for 7 days, they can be centrally stored and managed. At the same time, combined with the final diagnosis, some pure negative diagnoses can be deleted to save storage space and storage costs. During the piecing process, three layers of 200X (high magnification), 100X (medium magnification), and 40X (low magnification) will be generated. To save storage space, one of the magnification layer images can be selected for storage.

[0062] In addition to piecing, this embodiment can also collect the behavior trajectories of doctors when viewing slices, such as browsing and magnifying the slices, which is convenient for subsequent model training of AI artificial intelligence-assisted pathological slice viewing.

[0063] In a specific embodiment, the third embodiment of the present application provides a digital slide generation device based on a microscope, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the method described in any one of the second embodiments of the present application.

[0064] In a specific embodiment, the fourth embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the processor is caused to execute the steps of the method described in any one of the second embodiments of the present application.

[0065] Figure 5 The internal structure diagram of a computer device in an embodiment is shown. The computer device may specifically be a terminal or a server. Please refer to Figure 5 , this computer device includes a processor, a memory, etc. connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of this computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the method in this embodiment. A computer program may also be stored in the internal memory. When the computer program is executed by the processor, the processor can execute the method in this embodiment. Those skilled in the art can understand that Figure 5 the structure shown in

[0066] is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0067] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A microscope-based digital slide generation system, characterized in that, The system includes: a microscope, a shooting module, a positioning module, and a central control module; the shooting module is connected to the central control module, and the output end of the positioning module is connected to the input end of the central control module; When the object to be photographed is viewed through the microscope at different positions, the central control module is configured to send a shooting instruction to the shooting module and send a positioning instruction to the positioning module; The shooting module is configured to shoot digital slice images of the object to be photographed at different positions after being magnified by the microscope according to the shooting instruction, and send the digital slice images at different positions to the central control module; The positioning module is configured to obtain coordinate information of different positions when the object to be photographed is shot by the shooting module according to the positioning instruction, and send the coordinate information of different positions to the central control module; The central control module is configured to perform image fusion on the digital slice images at different positions and the coordinate information at different positions to obtain a microscopic image digital slice of the object to be photographed; The microscope includes objective lenses with different magnification factors. After shooting the digital slice images of the object to be photographed at different positions after being magnified by the microscope according to the shooting instruction, it further includes: Obtaining the magnification factors of the objective lenses corresponding to the digital slice images at different positions; Then, the performing image fusion on the digital slice images at different positions and the coordinate information at different positions to obtain a microscopic image digital slice of the object to be photographed includes: Performing image fusion on the digital slice images at different positions according to the magnification factor of the objective lens and the coordinate information at different positions to obtain a microscopic image digital slice of the object to be photographed; The performing image fusion on the digital slice images at different positions according to the magnification factor of the objective lens and the coordinate information at different positions to obtain a microscopic image digital slice of the object to be photographed includes: Constructing a movement curve of the object to be photographed according to the coordinate information at different positions; Performing image scaling on the digital slice images according to the magnification factors of the objective lenses corresponding to the digital slice images at different positions; wherein, all the scaled digital slice images have the same magnification factor; Fusing all the scaled digital slice images according to the coordinate information on the movement curve to obtain the microscopic image digital slice; The central control module is further configured to: When performing image fusion on the digital slice images at different positions, if there is an image area overlap between the first digital slice image and the second digital slice image, obtaining the first picture clarity of the image overlap area corresponding to the first digital slice image, and obtaining the second picture clarity of the image overlap area corresponding to the second digital slice image; the first digital slice image and the second digital slice image are any one of all the digital slice images; Comparing the first picture clarity and the second picture clarity; If the clarity of the first picture is greater than that of the second picture, select the fourth target digital slice image of the first digital slice image in the image coincidence area, and use the fourth target digital slice image as the image of the image coincidence area after image fusion; If the clarity of the first picture is less than that of the second picture, select the fifth target digital slice image of the second digital slice image in the image coincidence area, and use the fifth target digital slice image as the image of the image coincidence area after image fusion.

2. The microscope-based digital slide generation system according to claim 1, wherein, The positioning module obtains the Y-axis coordinate information in the coordinate information by detecting the distance between the preset fixed point of the stage moving platform of the microscope and the microscope wall; The X-axis coordinate information in the coordinate information is obtained by detecting the distance between the preset fixed point and the stage slice fixture of the microscope.

3. A method for generating digital slides based on a microscope, applied to the microscope-based digital slide generation system as described in claim 1, characterized in that, The method includes: When the object to be photographed is read at different positions through the microscope, send a photographing instruction to the photographing module and send a positioning instruction to the positioning module; Control the photographing module to photograph the digital slice images of the object to be photographed at different positions after being magnified by the microscope; Control the positioning module to obtain the coordinate information of different positions when the object to be photographed is photographed by the photographing module; Perform image fusion on the digital slice images of different positions and the coordinate information of different positions to obtain the microscopic image digital slice of the object to be photographed; If the microscope includes objective lenses with different magnifications, after controlling the photographing module to photograph the digital slice images of the object to be photographed at different positions after being magnified by the microscope, it further includes: Obtain the objective lens magnification corresponding to the digital slice images of different positions; Then the performing image fusion on the digital slice images of different positions and the coordinate information of different positions to obtain the microscopic image digital slice of the object to be photographed includes: Perform image fusion on the digital slice images of different positions according to the objective lens magnification and the coordinate information of different positions to obtain the microscopic image digital slice of the object to be photographed; The performing image fusion on the digital slice images of different positions according to the objective lens magnification and the coordinate information of different positions to obtain the microscopic image digital slice of the object to be photographed includes: Construct the movement curve of the object to be photographed according to the coordinate information of different positions; According to the objective lens magnification corresponding to the digital slice images of different positions, perform image scaling on the digital slice images; wherein, all the scaled digital slice images have the same magnification; On the movement curve, fuse all the scaled digital slice images according to the coordinate information to obtain the microscopic image digital slice; The method further includes: When performing image fusion on the digital slice images at different positions, if there is an overlapping image area between the first digital slice image and the second digital slice image, obtain the first picture clarity of the image overlapping area corresponding to the first digital slice image, and obtain the second picture clarity of the image overlapping area corresponding to the second digital slice image; the first digital slice image and the second digital slice image are any one of all the digital slice images; Compare the first picture clarity with the second picture clarity; If the first picture clarity is greater than the second picture clarity, select the fourth target digital slice image of the first digital slice image in the image overlapping area, and use the fourth target digital slice image as the image of the image overlapping area after image fusion; If the first picture clarity is less than the second picture clarity, select the fifth target digital slice image of the second digital slice image in the image overlapping area, and use the fifth target digital slice image as the image of the image overlapping area after image fusion.

4. The method for generating digital slides based on a microscope according to claim 3, characterized in that The objective lens magnification includes a first magnification, a second magnification, and a third magnification; the first magnification is less than the second magnification, and the second magnification is less than the third magnification; Then, performing image fusion on the digital slice images at different positions according to the objective lens magnification and the coordinate information at different positions to obtain the microscopic image digital slice of the object to be photographed includes: Construct a first layer corresponding to the first magnification, construct a second layer corresponding to the second magnification, and construct a third layer corresponding to the third magnification; Save the first target digital slice image obtained by using the first magnification to the first layer, save the second target digital slice image obtained by using the second magnification to the second layer, and save the third target digital slice image obtained by using the third magnification to the third layer; According to the coordinate information of the first target digital slice image, perform a puzzle operation on all the first target digital slice images in the first layer to obtain a first slice image; According to the coordinate information of the second target digital slice image, perform a puzzle operation on all the second target digital slice images in the second layer to obtain a second slice image; According to the coordinate information of the third target digital slice, perform a puzzle operation on all the third target digital slice images in the third layer to obtain a third slice image; Enlarge the first slice image according to the ratio of the second magnification to the first magnification to obtain a first simulated slice image; Retrieve a first target area where the image of the object to be photographed is not captured in the second slice image; Obtain first image information corresponding to the first target area in the first simulated slice image, and fill the first image information in the second slice image to obtain a second simulated slice image; According to the ratio of the third magnification factor to the second magnification factor, magnify the second simulated slice image to obtain a third simulated slice image; Retrieve a second target area where the image of the object to be photographed has not been collected in the third slice image; Obtain second image information corresponding to the second target area in the third simulated slice image, and fill the second image information in the third slice image to obtain the digital microscopic image slice.

5. The method for generating digital slides based on a microscope according to claim 3, characterized in that, The photographing of the digital slice images of the object to be photographed at different positions after being magnified by the microscope includes: Obtain the first field of view content of the object to be photographed after being magnified by the microscope, and photograph the first field of view content to obtain a digital slice image corresponding to the first field of view content; Move the object to be photographed so that the first field of view content changes to a second field of view content; the second field of view content is different from the first field of view content by a preset percentage of the field of view content; Photograph the second field of view content to obtain a digital slice image corresponding to the second field of view content, and use the second field of view content as the first field of view content, and return to the step of moving the object to be photographed so that the first field of view content changes to the second field of view content until all digital slice images corresponding to all field of view contents are obtained; all digital slice images corresponding to all field of view contents are the digital slice images at different positions.

6. A microscope-based digital slide generation device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to execute the steps of the method according to any one of claims 3 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the processor is caused to execute the steps of the method according to any one of claims 3 to 5.

Images