Microscope-based digital slice generation system, method and related equipment

Abstract

The embodiment of the invention discloses a digital slice generation system and method based on a microscope and related equipment, digital slice images corresponding to read films and coordinate information corresponding to the digital slice images are obtained while the microscope is used for reading the films, and the digital slice images are fused based on the coordinate information. Therefore, the digital slice image obtained during reading can be accurately fused into the microscopic image digital slice corresponding to the to-be-shot object according to the coordinate information, and a digital slice scanner does not need to be independently used for carrying out digital scanning operation on the to-be-shot object, so that the working process of digital slice generation is simplified, the diagnosis time is shortened, and the diagnosis efficiency is improved. And the diagnosis cost of pathology or other subjects using microscopes is reduced.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and in particular to a microscope-based digital slice generation system, method and related equipment. Background Art

[0002] Pathological diagnosis is the process of identifying diseases by analyzing slices of human tissues and cells. Microscopes are an indispensable tool in pathological diagnosis. Pathologists use ordinary optical microscopes to examine the microscopic structure of tissue samples taken from the lesion site through staining techniques (such as H&E staining), including cell morphology, arrangement and tissue structure, to determine whether there are abnormalities. Optical microscopes are also used for slice examinations, identification of the distribution and expression of specific types of proteins in tissues through immunohistochemical staining, and so on. In addition to optical microscopes, fluorescence microscopes can also mark specific molecules or structures with fluorescent dyes to detect infectious pathogens or specific gene expressions. In addition to clinical pathological diagnosis, microscopes are also widely used in microbiology, bone marrow images, neuroimmunology and other disciplines, but existing microscopes only have the functions of image magnification and image capture, and do not have the function of slice scanning. In order to obtain digital slices, doctors will perform separate digital scanning operations.

[0003] At present, the mainstream tool for digital slices is the "digital slice scanner", which has the advantage of high image clarity, but is expensive. Low-throughput scanners cannot scan large quantities of slices, while high-throughput scanners are still very expensive. In addition, adding a scanning link alone will increase a workflow, prolong the diagnosis time, and increase the diagnosis cost of pathology or other disciplines using microscopes. Summary of the invention

[0004] In view of this, the present invention provides a system, method and related equipment for generating digital slices based on a microscope.

[0005] The specific technical scheme of the first embodiment of the present invention is: a digital slice generation system based on a microscope, the system comprising: a microscope, a shooting module, a positioning module and a central control module; the shooting module is connected to the central control module, and the output end of the positioning module is connected to the input end of the central control module; When the object to be photographed is examined at different positions through the microscope, the central control module is used to send a shooting instruction to the shooting module, and send a positioning instruction to the positioning module; the shooting module is used to shoot digital slice images of the object to be photographed at different positions after being magnified by the microscope according to the shooting instruction, and send the digital slice images at different positions to the central control module; the positioning module is used to obtain the coordinate information of different positions of the object to be photographed when it is photographed by the shooting module according to the positioning instruction, and send the coordinate information of different positions to the central control module; the central control module is used to perform image fusion on the digital slice images at different positions and the coordinate information at different positions to obtain a digital slice of the microscopic image of the object to be photographed.

[0006] Preferably, the positioning module obtains the Y-axis coordinate information in the coordinate information by detecting the distance between a preset fixed point of the microscope's stage moving platform and the mirror wall of the microscope; and obtains the X-axis coordinate information in the coordinate information by detecting the distance between the preset fixed point and the microscope's stage slicing clamp.

[0007] The specific technical solution of the second embodiment of the present invention is: a microscope-based digital slice generation method, applied to the microscope-based digital slice generation system as described in the first embodiment of the present application, the method comprising: when the object to be photographed is read at different positions through the microscope, sending a shooting instruction to the shooting module, and sending a positioning instruction to the positioning module; controlling the shooting module to shoot digital slice images of the object to be photographed at different positions after being magnified by the microscope; controlling the positioning module to obtain the coordinate information of different positions of the object to be photographed when photographed by the shooting module; performing image fusion on the digital slice images at different positions and the coordinate information at different positions to obtain a microscopic image digital slice of the object to be photographed.

[0008] Preferably, the microscope includes objective lenses with different magnifications, and after controlling the shooting module to shoot the digital slice images of the object to be photographed at different positions after being magnified by the microscope, it also includes: obtaining the objective lens magnification corresponding to the digital slice images at different positions; and fusing the digital slice images at different positions with the coordinate information of the different positions to obtain the digital slices of the microscopic image of the object to be photographed, includes: fusing the digital slice images at different positions according to the objective lens magnification and the coordinate information of the different positions to obtain the digital slices of the microscopic image of the object to be photographed.

[0009] Preferably, the method of fusing the digital slice images at different positions according to the objective lens magnification and the coordinate information of the different positions to obtain the digital slice of the microscopic image of the object to be photographed includes: constructing a movement curve of the object to be photographed according to the coordinate information of the different positions; scaling the digital slice images according to the objective lens magnification corresponding to the digital slice images at different positions; wherein all scaled digital slice images have the same magnification; and fusing all scaled digital slice images on the movement curve according to the coordinate information to obtain the digital slice of the microscopic image.

[0010] Preferably, the objective lens magnification includes a first magnification, a second magnification and a third magnification; the first magnification is smaller than the second magnification, and the second magnification is smaller than the third magnification; then, according to the objective lens magnification and the coordinate information of the different positions, the digital slice images at the different positions are subjected to image fusion to obtain the microscopic image digital slice of the object to be photographed, including: constructing a first layer corresponding to the first magnification, constructing a second layer corresponding to the second magnification, and constructing a third layer corresponding to the third magnification; saving the first target digital slice image obtained by using the first magnification to the first layer, saving the second target digital slice image obtained by using the second magnification to the second layer, and saving the third target digital slice image obtained by using the third magnification to the third layer; according to the coordinate information of the first target digital slice image, performing a jigsaw operation on all the first target digital slice images in the first layer to obtain the first slice image; according to the coordinate information of the second target digital slice image, performing a jigsaw operation on all the first target digital slice images in the second layer to obtain the first slice image. performing a jigsaw operation on the second target digital slice image of the part to obtain a second slice image; performing a jigsaw operation on all the third target digital slice images in the third layer according to the coordinate information of the third target digital slice to obtain a third slice image; performing image amplification on the first slice image according to the ratio of the second magnification to the first magnification to obtain a first simulated slice image; retrieving a first target area of ​​the object to be photographed whose image is not collected in the second slice image; obtaining first image information corresponding to the first target area in the first simulated slice image, and filling the first image information in the second slice image to obtain a second simulated slice image; performing image amplification on the second simulated slice image according to the ratio of the third magnification to the second magnification to obtain a third simulated slice image; retrieving a second target area of ​​the object to be photographed whose image is not collected in the third slice image; obtaining second image information corresponding to the second target area in the third simulated slice image, and filling the second image information in the third slice image to obtain the microscopic image digital slice.

[0011] Preferably, the method further includes: when performing image fusion on the digital slice images at different positions, if the first digital slice image and the second digital slice image have overlapping image areas, obtaining a first picture clarity of the image overlapping area corresponding to the first digital slice image, and obtaining a second picture clarity of the image overlapping area corresponding to the second digital slice image; the first digital slice image and the second digital slice image are any one of all the digital slice images; comparing the first picture clarity and the second picture clarity; if the first picture clarity is greater than the second picture clarity, selecting a fourth target digital slice image of the first digital slice image in the image overlapping area, and using the fourth target digital slice image as the image of the image overlapping area after image fusion; if the first picture clarity is less than the second picture clarity, selecting a fifth target digital slice image of the second digital slice image in the image overlapping area, and using the fifth target digital slice image as the image of the image overlapping area after image fusion.

[0012] Preferably, the step of obtaining digital slice images of different positions of the object to be photographed after being magnified by a microscope includes: obtaining a first field of view content of the object to be photographed after being magnified by the microscope, and photographing the first field of view content to obtain a digital slice image corresponding to the first field of view content; moving the object to be photographed so that the first field of view content is changed to a second field of view content; the second field of view content is different from the first field of view content by a preset percentage; photographing the second field of view content to obtain a digital slice image corresponding to the second field of view content, and using the second field of view content as the first field of view content, returning to the step of moving the object to be photographed so that the first field of view content is changed to the second field of view content, until all digital slice images corresponding to all field of view contents are obtained; and all digital slice images corresponding to all field of view contents are the digital slice images at the different positions.

[0013] The specific technical solution of the third embodiment of the present invention is: a microscope-based digital slice generation device, including a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method described in any one of the second embodiments of the present application.

[0014] The specific technical solution of the fourth embodiment of the present invention is: a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the method described in any one of the second embodiments of the present application.

[0015] Implementing the embodiments of the present invention will have the following beneficial effects: The present invention obtains a digital slice image corresponding to the image reading and coordinate information corresponding to the digital slice image while reading the image under a microscope, and fuses the digital slice image based on the coordinate information, so that the digital slice image obtained during the image reading can be accurately fused into a microscopic image digital slice corresponding to the object to be photographed according to the coordinate information, without the need to use a digital slice scanner separately to perform a digital scanning operation on the object to be photographed, thereby simplifying the workflow of digital slice generation, reducing diagnosis time, and reducing the diagnosis cost of pathology or other disciplines using microscopes. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A schematic diagram of the structure of the system for generating digital slides based on microscope; Figure 2 A flowchart of the steps of a first embodiment of a method for generating digital slices based on a microscope; Figure 3 It is a flowchart of the steps of image fusion based on coordinate information; Figure 4 A flowchart of the steps of a second embodiment of a method for generating digital slices based on a microscope; Figure 5 It is a diagram of the internal structure of a computer device; Among them, 101 is a microscope; 102 is a shooting module; 103 is a positioning module; 104 is a central control module. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] The terms "first", "second", etc. in the specification, claims and drawings of this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other steps or modules that are inherent to these processes, methods, products or devices.

[0020] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] See also Figure 1 , is a structural schematic diagram of a digital slice generation system based on a microscope in the first embodiment of the present application, in order to simplify the workflow of digital slice generation and reduce the diagnosis time, the system includes: a microscope 101, a shooting module 102, a positioning module 103 and a central control module 104; the shooting module 102 is connected to the central control module 104, and the output end of the positioning module 103 is connected to the input end of the central control module 104; when the object to be photographed is read at different positions through the microscope, the central control module is used to send a shooting instruction to the shooting module, and send a positioning instruction to the positioning module; the shooting module is used to shoot digital slice images of the object to be photographed at different positions after being magnified by the microscope according to the shooting instruction, and send the digital slice images at different positions to the central control module; the positioning module is used to obtain the coordinate information of different positions of the object to be photographed when it is photographed by the shooting module according to the positioning instruction, and send the coordinate information of different positions to the central control module; the central control module is used to perform image fusion on the digital slice images at different positions and the coordinate information at different positions to obtain a microscopic image digital slice of the object to be photographed.

[0022] Specifically, the shooting module can be an electronic objective lens, and the positioning module can be a distance measuring component such as a laser distance sensor or an optical ruler. The positioning module identifies the current coordinate information of the microscope stage and transmits it to the central control module in real time. The optical ruler sensor is integrated into the X / Y axis guide rail of the stage and synchronized with the moving slider. The laser sensor is external, fixed to the stage base, pointing to the reflective surface of the slicing fixture to ensure that it is not disturbed by vibration during use.

[0023] When the object to be photographed is examined at different positions through a microscope, the central control module sends a shooting instruction to the shooting module, and sends a positioning instruction to the positioning module; the shooting module shoots digital slice images of the object to be photographed at different positions after being magnified by the microscope according to the shooting instruction. For example, when the magnification of the current microscope is 40X, the shooting module will shoot digital slice images of the object to be photographed at different positions after being magnified by 40X, and send the digital slice images of the object to be photographed after being magnified by 40X to the central control module; the positioning module obtains coordinate information of different positions according to the positioning instruction, and sends the coordinate information to the central control module; the central control module performs image fusion on the digital slice images of different positions and the coordinate information of different positions to obtain a digital slice of the microscopic image of the object to be photographed.

[0024] The system in this embodiment obtains the digital slice image corresponding to the reading and the coordinate information corresponding to the digital slice image while reading the film using a microscope, and fuses the digital slice image based on the coordinate information. This can achieve accurate fusion of the digital slice image obtained during the reading into a microscopic image digital slice corresponding to the object to be photographed according to the coordinate information. There is no need to use a digital slice scanner separately to perform digital scanning operations on the object to be photographed, thereby simplifying the workflow of digital slice generation, reducing diagnosis time, and reducing the diagnosis cost of pathology or other disciplines using microscopes.

[0025] In a specific embodiment, the positioning module obtains the Y-axis coordinate information in the coordinate information by detecting the distance between a preset fixed point of the microscope's stage moving platform and the mirror wall of the microscope; and obtains the X-axis coordinate information in the coordinate information by detecting the distance between the preset fixed point and the microscope's stage slicing fixture.

[0026] Specifically, when the positioning module is a laser distance sensor, the laser distance sensor is set at the upper left corner of the stage moving platform, the distance between the laser distance sensor and the mirror wall is the Y-axis coordinate information, and the distance between the laser distance sensor and the stage slicing fixture is the X-axis coordinate information. It is also possible to set a reflective point relative to the position of the laser distance sensor in the direction where the laser distance sensor measures the Y-axis coordinate information, and the distance between the laser distance sensor and the reflective point is the Y-axis coordinate information.

[0027] In the specific embodiments, see Figure 2 The second embodiment of the present application provides a flowchart of the steps of a method for generating digital slices based on a microscope, which is applied to the system for generating digital slices based on a microscope as described in the first embodiment of the present application, and the method includes: Step 201, when the object to be photographed is read at different positions through the microscope, a photographing instruction is sent to the photographing module, and a positioning instruction is sent to the positioning module; Step 202, controlling the shooting module to shoot digital slice images of the object to be photographed at different positions after being magnified by the microscope; Step 203: Control the positioning module to obtain coordinate information of different positions of the object to be photographed when photographed by the photographing module; Step 204: performing image fusion on the digital slice images at different positions and the coordinate information at different positions to obtain a microscopic image digital slice of the object to be photographed.

[0028] Specifically, when the object to be photographed is examined at different positions through a microscope, a shooting instruction is sent to the shooting module, and a positioning instruction is sent to the positioning module; the shooting module is controlled to shoot digital slice images of the object to be photographed at different positions after being magnified by the microscope according to the shooting instruction. For example, when the magnification of the current microscope is 40X, the shooting module will shoot digital slice images of the object to be photographed at different positions after being magnified by 40X, and the positioning module is controlled to obtain coordinate information of different positions, and the digital slice images at different positions and the coordinate information at different positions are image-fused to obtain digital slices of the microscopic image of the object to be photographed, so as to achieve accurate fusion of the digital slice images obtained during the examination into digital slices of the microscopic image corresponding to the object to be photographed according to the coordinate information, without the need to use a digital slice scanner separately to perform digital scanning operations on the object to be photographed, thereby simplifying the workflow of digital slice generation, reducing diagnosis time, and reducing the diagnosis cost of pathology or other disciplines using microscopes.

[0029] In a specific embodiment, the microscope includes objective lenses with different magnifications. Then, after controlling the shooting module to shoot the digital slice images of the object to be photographed at different positions after being magnified by the microscope, it also includes: obtaining the objective lens magnification corresponding to the digital slice images at different positions; then, fusing the digital slice images at different positions and the coordinate information of the different positions to obtain the digital slices of the microscopic image of the object to be photographed includes: fusing the digital slice images at different positions according to the objective lens magnification and the coordinate information of the different positions to obtain the digital slices of the microscopic image of the object to be photographed.

[0030] Specifically, the microscope will be provided with objective lenses of different magnifications, such as a 40X low-power lens, a 100X medium-power lens, or a 200X high-power lens. When the user uses the microscope to read the film, the objective lenses of different magnifications will be used for reading according to the needs. For example, when reading the A area of ​​the object to be photographed, a 40X low-power lens is used for reading, and when reading the B area of ​​the object to be photographed, a 100X medium-power lens is used for reading. Therefore, the objective lens magnification of the obtained digital slice image will be inconsistent. To this end, the objective lens magnification corresponding to each digital slice image will be obtained first, and then the digital slice images at different positions will be fused according to the objective lens magnification and the coordinate information of different positions to obtain the digital slice of the microscopic image of the object to be photographed, thereby realizing the accurate fusion of digital slice images with different objective lens magnifications.

[0031] In a specific embodiment, the objective lens magnification is obtained in the following manner: 1. Objective lens switching sensor: A Hall sensor is used. When a magnet is embedded in the objective lens turntable, a Hall signal is triggered when the objective lens is in place. The central control module collects the Hall signal. Different Hall signals represent different objective lens magnifications. Alternatively, a photoelectric sensor is used to detect the notch position of the objective lens turntable. Different notch positions represent different objective lens magnifications.

[0032] 2. Magnification coding mapping: Encode the objective lenses of different magnifications, for example, the 40X objective lens corresponds to the code 01, the 100X objective lens corresponds to the code 10, and the 200X objective lens corresponds to the code 11. The central processing module identifies the corresponding code to obtain the objective lens magnification.

[0033] In the specific embodiments, see Figure 3 , the step of fusing the digital slice images at different positions according to the objective lens magnification and the coordinate information of the different positions to obtain the digital slice of the microscopic image of the object to be photographed comprises: Step 301: constructing a movement curve of the object to be photographed according to the coordinate information of the different positions; Step 302, scaling the digital slice images according to the objective lens magnifications corresponding to the digital slice images at different positions; wherein all scaled digital slice images have the same magnification; Step 303: On the moving curve, all scaled digital slice images are fused according to the coordinate information to obtain the microscopic image digital slice.

[0034] Specifically, the function of the moving curve is to initially place a large number of digital slice images on the puzzle blackboard, so as to facilitate the selection of digital slice images above, below, left and right in subsequent calculations. In addition, when the digital slice images do not overlap, the discontinuous digital slice images can be placed in a roughly correct position. After the digital slice images are placed on the moving curve according to the center point, each image is marked as an adjacent image with the center point coordinates as the center of the circle, and the center point coordinate images within a radius of 1 times the major axis are retrieved. First, the digital slice images of different magnifications are scaled and adjusted to digital slice images of the same magnification, such as images that are all scaled to 40X or 100X, etc. For images at the same magnification, the relative displacement of the adjacent images and the current image of each image is calculated based on the actual moving coordinates of the moving curve. The relative displacement is adjusted more accurately by calculating the overlapping area, and microscopic stitching is performed at the cellular level.

[0035] In a specific embodiment, the method further includes: when performing image fusion on the digital slice images at different positions, if the first digital slice image and the second digital slice image have an overlapping image area, obtaining a first picture clarity of the image overlapping area corresponding to the first digital slice image, and obtaining a second picture clarity of the image overlapping area corresponding to the second digital slice image; the first digital slice image and the second digital slice image are any one of all the digital slice images; comparing the first picture clarity and the second picture clarity; if the first picture clarity is greater than the second picture clarity, selecting a fourth target digital slice image of the first digital slice image in the image overlapping area, and using the fourth target digital slice image as the image of the image overlapping area after image fusion; if the first picture clarity is less than the second picture clarity, selecting a fifth target digital slice image of the second digital slice image in the image overlapping area, and using the fifth target digital slice image as the image of the image overlapping area after image fusion.

[0036] In a specific embodiment, for the repeated area, a preset software algorithm is used to read the clarity of different layers in the repeated area and perform numerical quantification. The image with a higher clarity value is selected as the display image of the overlapping area, and the remaining images are deleted to increase the running speed of image fusion. The whole image is alternately scanned to finally achieve clear scanning of most areas.

[0037] Specifically, the clarity of an image is measured by calculating the edge strength (i.e., clarity) of the image, using the Sobel operator and gradient calculation to detect the edge of the image, and quantifying the clarity of the image by calculating the edge strength.

[0038] For example, the Sobel operator calculates the gradient: gray=cv2.cvtColor(image, cv2.COLOR_BGR2GRAY); sobel_x = cv2.Sobel(gray, cv2.CV_64F, 1, 0, ksize=3); sobel_y = cv2.Sobel(gray, cv2.CV_64F, 0, 1, ksize=3).

[0039] For example, the mean of the gradient magnitude is used as a clarity indicator: magnitude = np.sqrt(sobel_x2 + sobel_y2); return np.mean(magnitude). If the edge strength of the image is large, it indicates that the image clarity is high and the details are clearly visible. If the image edge is fuzzy, it indicates that the image clarity is low and blurry.

[0040] In a specific embodiment, the objective lens magnification includes a first magnification, a second magnification and a third magnification; the first magnification is smaller than the second magnification, and the second magnification is smaller than the third magnification; then, according to the objective lens magnification and the coordinate information of the different positions, the digital slice images at the different positions are subjected to image fusion to obtain the microscopic image digital slice of the object to be photographed, including: constructing a first layer corresponding to the first magnification, constructing a second layer corresponding to the second magnification, and constructing a third layer corresponding to the third magnification; saving the first target digital slice image obtained by using the first magnification to the first layer, saving the second target digital slice image obtained by using the second magnification to the second layer, and saving the third target digital slice image obtained by using the third magnification to the third layer; according to the coordinate information of the first target digital slice image, performing a jigsaw operation on all the first target digital slice images in the first layer to obtain the first slice image; according to the coordinate information of the second target digital slice image, the second layer is combined into a jigsaw puzzle to obtain the first slice image. performing a jigsaw operation on all the second target digital slice images in the third layer to obtain a second slice image; performing a jigsaw operation on all the third target digital slice images in the third layer according to the coordinate information of the third target digital slice to obtain a third slice image; performing image amplification on the first slice image according to the ratio of the second magnification to the first magnification to obtain a first simulated slice image; retrieving a first target area of ​​the object to be photographed, whose image has not been collected, in the second slice image; obtaining first image information corresponding to the first target area in the first simulated slice image, and filling the first image information in the second slice image to obtain a second simulated slice image; performing image amplification on the second simulated slice image according to the ratio of the third magnification to the second magnification to obtain a third simulated slice image; retrieving a second target area of ​​the object to be photographed, whose image has not been collected, in the third slice image; obtaining second image information corresponding to the second target area in the third simulated slice image, and filling the second image information in the third slice image to obtain the microscopic image digital slice.

[0041] Specifically, the objective lens magnifications include 40X, 100X, and 200X, and a first layer corresponding to 40X, a second layer corresponding to 100X, and a third layer corresponding to 200X are constructed respectively; the first target digital slice image acquired at 40X is saved in the first layer, the second target digital slice image acquired at 100X is saved in the second layer, and the third target digital slice image acquired at 200X is saved in the third layer. According to the coordinate information of the first target digital slice image in the first layer, all the first target digital slice images in the first layer are jigsawed to obtain the first slice image; according to the coordinate information of the second target digital slice image in the second layer, all the second target digital slice images in the second layer are jigsawed to obtain the second slice image; according to the coordinate information of the third target digital slice in the third layer, all the third target digital slice images in the third layer are jigsawed to obtain the third slice image. The first slice image magnified at 40X is magnified by 2.5 times to generate a first simulated slice image simulating 100X, and the visual field area that has not been collected is retrieved in the second slice image, and the area is filled with the first simulated slice image simulating 100X to generate a second simulated slice image of 100X. The second simulated slice image of 100X is magnified by 2 times to generate a third simulated slice image simulating 200X, and the visual field area that has not been collected in the third slice image magnified by 200X is retrieved, and the area is filled with the third simulated slice image simulating 200X. The images after the two filling operations are stored to generate a general digital slice image in JPG format.

[0042] In a specific embodiment, the step of obtaining digital slice images of different positions of the object to be photographed after being magnified by a microscope includes: obtaining a first field of view content of the object to be photographed after being magnified by the microscope, and photographing the first field of view content to obtain a digital slice image corresponding to the first field of view content; moving the object to be photographed so that the first field of view content is changed to a second field of view content; the second field of view content is different from the first field of view content by a preset percentage; photographing the second field of view content to obtain a digital slice image corresponding to the second field of view content, and using the second field of view content as the first field of view content, returning to the step of moving the object to be photographed so that the first field of view content is changed to the second field of view content, until all digital slice images corresponding to all field of view contents are obtained; and all digital slice images corresponding to all field of view contents are the digital slice images at the different positions.

[0043] Specifically, according to different magnifications, the preset values ​​under three magnifications of 200X (high magnification), 100X (medium magnification), and 40X (low magnification) are set respectively. The setting standard is that the single-direction movement distance accounts for 50% of the current field of view width. If the movement in any direction of the X-axis or Y-axis exceeds the preset value, it is judged as effective movement. The calculation formula for the field of view width is: field of view width = field of view size of the objective lens magnification sensor / objective lens magnification. For example, the CMOS size of the objective lens magnification sensor is 22mm, and the field of view width of the 200X objective lens = 22mm / 200 = 0.11mm); the movement trigger threshold of 200X = 0.055mm (0.11mm×50%); the movement trigger threshold of 100X = 0.11mm; the movement trigger threshold of 40X = 0.275mm; when it is greater than the movement trigger threshold, the electronic objective lens is triggered to collect a real-time microscopic image of the field of view under the front mirror. Trigger conditions:

[0044] in, is the X coordinate value of the current coordinate point, is the Y coordinate value of the current coordinate point, is the X coordinate value of the coordinate point collected last time, It is the Y coordinate value of the coordinate point collected last time. After effective movement, the electronic objective lens is triggered to take pictures. The reason for not taking pictures continuously is that the high-speed camera will generate a lot of information when taking pictures continuously, which will cause the system to run slowly.

[0045] In a specific embodiment, the complete step diagram of the present application is as follows: Figure 4 As shown. In the process of digital slicing, first obtain the coordinate position information and collect the magnification information of the microscope objective lens. Under different magnifications, calculate the coordinate movement distance. When the movement distance exceeds the preset value (50% of the field of view width at the current magnification), the central control module issues a command to trigger the camera module to take a photo. Combined with the identified absolute coordinate displacement, the slice movement direction information and movement distance are calculated. At the same time, the AI ​​algorithm is combined to accurately identify the more accurate movement direction and movement distance of the image under the microscope, and the two information are double-jigsawed. For the slices of the day, it can be set to local storage, and for the slices that have been diagnosed for 7 days, they can be centrally stored and managed. At the same time, combined with the final diagnosis, some pure negative diagnoses can be deleted to save storage space and storage costs. During the jigsaw process, three layers of 200X (high magnification), 100X (medium magnification), and 40X (low magnification) will be generated. In order to save storage space, one of the magnification layer images can be selected for storage.

[0046] In addition to puzzle-solving, this embodiment can also collect the doctor's behavior trajectory of browsing and zooming in on slices when reading films, which is convenient for subsequent AI artificial intelligence-assisted pathology film reading model training.

[0047] In a specific embodiment, the third embodiment of the present application provides a microscope-based digital slice generation device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described in any one of the second embodiments of the present application.

[0048] In a specific embodiment, the fourth embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the method as described in any one of the second embodiments of the present application.

[0049] Figure 5 The internal structure of a computer device in one embodiment is shown. The computer device can be a terminal or a server. Figure 5 The computer device includes a processor, a memory, etc. connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the method in this embodiment. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement the method in this embodiment. Those skilled in the art can understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0050] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims. The above description is only a preferred embodiment of the present invention and does not limit the present invention in other forms. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A digital slice generation system based on a microscope, characterized in that: The system comprises: a microscope, a shooting module, a positioning module and a central control module; the shooting module is connected to the central control module, and the output end of the positioning module is connected to the input end of the central control module; When the object to be photographed is read at different positions through the microscope, the central control module is used to send a photographing instruction to the photographing module and send a positioning instruction to the positioning module; The shooting module is used to shoot digital slice images of the object to be photographed at different positions after being magnified by the microscope according to the shooting instruction, and send the digital slice images at different positions to the central control module; The positioning module is used to obtain the coordinate information of different positions of the object to be photographed when photographed by the photographing module according to the positioning instruction, and send the coordinate information of the different positions to the central control module; The central control module is used to perform image fusion on the digital slice images at different positions and the coordinate information at different positions to obtain a digital slice of a microscopic image of the object to be photographed; The microscope includes objective lenses with different magnifications, and after the digital slice images of the object to be photographed at different positions after being magnified by the microscope are photographed according to the photographing instruction, the method further includes: Obtain the objective lens magnification corresponding to the digital slice images at different positions; Then, the step of fusing the digital slice images at different positions and the coordinate information at different positions to obtain the digital slice of the microscopic image of the object to be photographed includes: According to the objective lens magnification and the coordinate information of the different positions, the digital slice images at the different positions are image-fused to obtain a digital slice of the microscopic image of the object to be photographed; The step of fusing the digital slice images at different positions according to the objective lens magnification and the coordinate information of the different positions to obtain the digital slice of the microscopic image of the object to be photographed comprises: Constructing a movement curve of the object to be photographed according to the coordinate information of the different positions; Scaling the digital slice images according to the objective lens magnifications corresponding to the digital slice images at different positions; wherein all scaled digital slice images have the same magnification; On the moving curve, all scaled digital slice images are merged according to coordinate information to obtain the microscopic image digital slice; The central control module is also used to: When the digital slice images at different positions are image-fused, if the first digital slice image and the second digital slice image have overlapping image regions, obtaining a first picture clarity of the overlapping image region corresponding to the first digital slice image, and obtaining a second picture clarity of the overlapping image region corresponding to the second digital slice image; the first digital slice image and the second digital slice image are any one of all the digital slice images; comparing the definition of the first picture with the definition of the second picture; If the definition of the first image is greater than that of the second image, selecting a fourth target digital slice image of the first digital slice image in the image overlap area, and using the fourth target digital slice image as the image of the image overlap area after image fusion; If the definition of the first image is smaller than that of the second image, a fifth target digital slice image of the second digital slice image in the image overlap area is selected, and the fifth target digital slice image is used as the image of the image overlap area after image fusion.

2. The microscope-based digital slice generation system according to claim 1, characterized in that: The positioning module obtains the Y-axis coordinate information in the coordinate information by detecting the distance between a preset fixed point of the stage moving platform of the microscope and the mirror wall of the microscope; The X-axis coordinate information in the coordinate information is obtained by detecting the distance between the preset fixed point and the stage slice fixture of the microscope.

3. A method for generating digital slices based on a microscope, applied to the system for generating digital slices based on a microscope as claimed in claim 1, characterized in that: The method comprises: When the object to be photographed is read at different positions through the microscope, a photographing instruction is sent to the photographing module, and a positioning instruction is sent to the positioning module; Controlling the shooting module to shoot digital slice images of the object to be photographed at different positions after being magnified by the microscope; Controlling the positioning module to obtain coordinate information of different positions of the object to be photographed when photographed by the photographing module; Performing image fusion on the digital slice images at different positions and the coordinate information at different positions to obtain a digital slice of a microscopic image of the object to be photographed; The microscope includes objective lenses with different magnifications, and after controlling the shooting module to shoot digital slice images of the object to be shot at different positions after being magnified by the microscope, the method further includes: Obtain the objective lens magnification corresponding to the digital slice images at different positions; Then, the step of fusing the digital slice images at different positions and the coordinate information at different positions to obtain the digital slice of the microscopic image of the object to be photographed includes: According to the objective lens magnification and the coordinate information of the different positions, the digital slice images at the different positions are image-fused to obtain a digital slice of the microscopic image of the object to be photographed; The step of fusing the digital slice images at different positions according to the objective lens magnification and the coordinate information of the different positions to obtain the digital slice of the microscopic image of the object to be photographed comprises: Constructing a movement curve of the object to be photographed according to the coordinate information of the different positions; Scaling the digital slice images according to the objective lens magnifications corresponding to the digital slice images at different positions; wherein all scaled digital slice images have the same magnification; On the moving curve, all scaled digital slice images are merged according to coordinate information to obtain the microscopic image digital slice; The method further comprises: When the digital slice images at different positions are image-fused, if the first digital slice image and the second digital slice image have overlapping image regions, obtaining a first picture clarity of the overlapping image region corresponding to the first digital slice image, and obtaining a second picture clarity of the overlapping image region corresponding to the second digital slice image; the first digital slice image and the second digital slice image are any one of all the digital slice images; comparing the definition of the first picture with the definition of the second picture; If the definition of the first image is greater than that of the second image, selecting a fourth target digital slice image of the first digital slice image in the image overlap area, and using the fourth target digital slice image as the image of the image overlap area after image fusion; If the definition of the first image is smaller than that of the second image, a fifth target digital slice image of the second digital slice image in the image overlap area is selected, and the fifth target digital slice image is used as the image of the image overlap area after image fusion.

4. The method for generating digital slices based on a microscope according to claim 3, characterized in that: The objective lens magnification includes a first magnification, a second magnification and a third magnification; the first magnification is smaller than the second magnification, and the second magnification is smaller than the third magnification; Then, the step of fusing the digital slice images at different positions according to the objective lens magnification and the coordinate information of the different positions to obtain the microscopic image digital slice of the object to be photographed includes: constructing a first layer corresponding to the first magnification, constructing a second layer corresponding to the second magnification, and constructing a third layer corresponding to the third magnification; The first target digital slice image acquired by the first magnification is saved in the first layer, the second target digital slice image acquired by the second magnification is saved in the second layer, and the third target digital slice image acquired by the third magnification is saved in the third layer; According to the coordinate information of the first target digital slice image, all the first target digital slice images in the first layer are subjected to a jigsaw operation to obtain a first slice image; According to the coordinate information of the second target digital slice image, all the second target digital slice images in the second layer are subjected to a jigsaw operation to obtain a second slice image; According to the coordinate information of the third target digital slice, all the third target digital slice images in the third layer are subjected to a jigsaw operation to obtain a third slice image; According to the ratio of the second magnification factor to the first magnification factor, the first slice image is magnified to obtain a first simulated slice image; Retrieving a first target area of ​​the object to be photographed in which no image is acquired in the second slice image; Acquire first image information corresponding to the first target area in the first simulated slice image, and fill the first image information in the second slice image to obtain a second simulated slice image; Amplifying the second simulated slice image according to a ratio of the third magnification factor to the second magnification factor to obtain a third simulated slice image; Retrieving a second target area of ​​the object to be photographed in which no image is acquired in the third slice image; The second image information corresponding to the second target area is acquired in the third simulated slice image, and the second image information is filled in the third slice image to obtain the microscopic image digital slice.

5. The method for generating digital slices based on a microscope according to claim 3, characterized in that: The step of photographing digital slice images of the object to be photographed at different positions after being magnified by the microscope comprises: Acquiring a first field of view content of the object to be photographed after being magnified by the microscope, and photographing the first field of view content to obtain a digital slice image corresponding to the first field of view content; Moving the object to be photographed so that the first field of view content is changed to a second field of view content; the second field of view content is different from the first field of view content by a preset percentage; The second field of view content is photographed to obtain a digital slice image corresponding to the second field of view content, and the second field of view content is used as the first field of view content, and the step of moving the object to be photographed to change the first field of view content to the second field of view content is returned to, until all digital slice images corresponding to all field of view contents are obtained; and all digital slice images corresponding to all field of view contents are the digital slice images at different positions.

6. A digital slice generation device based on a microscope, comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 3 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 3 to 5.

Images