Microscopic Automatic Cutting Method and Equipment for Tumor Cell Sampling

Abstract

The present invention discloses a microscopic automatic cutting method and device for tumor cell sampling, comprising: acquiring a WSI image; identifying all tumor regions and their positions on the WSI image; acquiring an initial image using a visual monitoring device; obtaining the tumor regions and their positions on the initial image; identifying the position of the cutting needle tip on the initial image; calculating through a path planning algorithm to obtain the cutting needle movement path; performing cutting on the glass slide; during the cutting process, using the visual monitoring device to acquire a real-time image and comparing the real-time image with the initial image to obtain the deformation amount of the normal tissue region, and when the deformation amount exceeds the threshold, stopping the cutting; otherwise, continuously cutting until the current tumor region is cut; after the current tumor region is cut, moving the glass slide and repeating the above steps until all tumor regions on the glass slide are cut. The present invention can accurately identify the positions of tumor regions, achieving high-precision positioning and fully automatic cutting.

Description

Technical Field

[0001] The invention belongs to the field of biomedical technology, and in particular relates to a microscopic automatic cutting method and equipment for tumor cell sampling. Background Art

[0002] Isolating the desired cell population from tissue sections or cell smears is essential for subsequent molecular biological analysis. Currently, laser capture microdissection and mechanical microdissection are two commonly used cell isolation methods.

[0003] Laser capture microdissection technology uses a laser beam to shoot vertically from above the slice, and the light spot falls on the area to be cut. When the film in the area is lifted, the cells connected to it are also cut off. This technology has the advantages of precise cutting, flexible operation, good sample integrity, and can be used in conjunction with a variety of analytical techniques. However, it also has disadvantages such as high equipment cost, high requirements for operators, relatively low cutting flux, and the risk of thermal damage.

[0004] Mechanical microdissection technology uses special micro-knives, blades or cutting needles to directly cut tissues or cells under a microscope. This technology is simple and intuitive, relatively low-cost and easy to understand. However, the cutting process is manually operated, and the cutting accuracy is affected by individual differences between operators. It is difficult to handle tiny, delicate and dispersed cell groups, and the cutting efficiency is low.

[0005] In addition, whether it is laser capture microdissection or mechanical microdissection, there is a balance between field of view and efficiency. If a higher magnification is used, the display field of view will be reduced and the efficiency of the operation will be low. If a lower magnification is used, although the field of view is expanded, the magnification of the image is too low, which will affect the accuracy of observation.

[0006] The existing laser capture microdissection technology has the following problems:

[0007] First, the existing laser capture microdissection technology equipment is expensive, including the instrument itself, laser light source, high-precision microscope, professional control software, etc., which all require high investment, and the subsequent maintenance costs are also high, which limits its popularization and application in some laboratories or scientific research institutions with limited budgets.

[0008] Second, laser capture microdissection technology has high requirements for operators, who need to be proficient in microscope operation skills, laser parameter settings, and sample processing procedures. It requires professional and long-term training to get started, and the operation requirements are high.

[0009] Third, the cutting throughput of laser capture microdissection technology is relatively low, and it can only process samples in a small area at a time. It is inefficient for large-scale, high-throughput sample collection tasks.

[0010] Fourth, the laser capture microdissection technology has a certain risk of thermal damage. When used for a long time and at high intensity, it may cause thermal damage to the sample, affect the activity of intracellular biological macromolecules, and interfere with the subsequent molecular biology test results.

[0011] Fifth, the mechanical microdissection technology involves manual operation during the cutting process. The cutting accuracy is affected by individual differences of operators, making it difficult to process small, delicate, and dispersed cell populations, resulting in low cutting efficiency.

[0012] Therefore, there is an urgent need for a new type of microdissection technology that can maintain the advantages of high-precision cutting, flexible operation, and good sample integrity, while reducing equipment costs and improving automation, so as to meet the requirements of biomedical research for high-quality biological samples. Summary of the Invention

[0013] To solve the above technical problems, the present invention proposes a micro-automatic cutting method and device for tumor cell sampling.

[0014] To achieve the above object, the technical solution of the present invention is as follows:

[0015] On the one hand, the present invention discloses a micro-automatic cutting method for tumor cell sampling, including:

[0016] Step S1: Scanning a glass slide using a digital microscope to obtain a WSI image;

[0017] Step S2: Using a pre-trained tumor recognition model to identify all tumor regions and their positions on the WSI image;

[0018] Step S3: Placing one or more glass slides on and fixing them on a stage, and using one or more visual monitoring devices to obtain an initial image I0, on which a section and a cutting needle are presented;

[0019] Step S4: Determining the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image through an image registration algorithm, and obtaining the tumor regions and their positions on the initial image I0;

[0020] Step S5: Identifying the position of the needle tip of the cutting needle on the initial image I0 through a needle tip recognition algorithm;

[0021] Step S6: Taking the position of the needle tip of the cutting needle on the initial image I0 as the initial position and the position of the tumor region to be cut on the initial image I0 as the target position, and calculating through a path planning algorithm to obtain the movement path of the cutting needle;

[0022] Step S7: After the cutting needle moves to the target position according to the obtained movement path of the cutting needle, it drops and cuts on the corresponding glass slide.

[0023] During the cutting process, a visual monitoring device is used to obtain a real-time image I t , and the real-time image I t is compared with the initial image I0 to obtain the deformation amount of the normal tissue area,

[0024] When the deformation amount exceeds the threshold, stop cutting; otherwise, keep cutting until the current tumor area is completely cut;

[0025] Step S8: After the current tumor area is completely cut, move the glass slide, and then repeat steps S3 - S7 until all tumor areas on the glass slide are completely cut, realizing tumor cell sampling.

[0026] Based on the above technical solution, the following improvements can also be made:

[0027] As a preferred solution, step S4 includes:

[0028] Step S4.1: Using the initial image I0 as the reference image and the WSI image as the floating image, through the image registration algorithm, register the initial image I0 with the WSI image;

[0029] Step S4.2: After image registration, determine the position of the initial image I0 on the WSI image, and obtain the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image;

[0030] Step S4.3: According to the relative relationship between the two coordinate systems, based on all tumor areas and their positions on the WSI image, obtain the tumor areas and their positions on the initial image I0.

[0031] As a preferred solution, step S5 includes:

[0032] Step S5.1: Using the needle tip image of the cutting needle as the template image, slide pixel by pixel on the initial image I0, and use the similarity measurement method to calculate and record the matching degree between the template image and the sliding area of the initial image I0 at each position;

[0033] Step S5.2: Find the position with the highest matching degree, which is the position of the cutting needle on the initial image I0.

[0034] As a preferred solution, step S7 includes:

[0035] Step S7.1: During the cutting process, a visual monitoring device is used to obtain a real-time image I t , and obtain the real-time normal tissue area N according to the following formula t :

[0036] N t = I t- T0;

[0037] where: T0 is the tumor region on the initial image I0;

[0038] Step S7.2: Obtain the initial normal tissue region N0 according to the following formula:

[0039] N0 = I0 - T0;

[0040] Step S7.3: Calculate the average value of the sum of the squares of the differences between the pixel values of the real-time normal tissue region N t and the corresponding pixel values of the initial normal tissue region N0, obtain the deformation amount of the normal tissue region, and compare it with the threshold. When the deformation amount exceeds the threshold, stop cutting;

[0041] Step S7.4: Repeat Step S7.1 - Step S7.3 until the current tumor region is completely cut.

[0042] As a preferred solution, Step S6 includes:

[0043] Step S6.1: Extract features from a part of the tumor region to be cut to obtain tumor features;

[0044] Step S6.2: Use the trained tumor complexity judgment model to judge the complexity of the tumor region to be cut, and output the complexity level T and the number of cutting needles n;

[0045] Step S6.3: Select the n cutting needles closest to the tumor region to be cut as the cutting needles to be actuated, and allocate corresponding cutting regions to each cutting needle to be actuated;

[0046] Step S6.4: Take the needle tip position of each cutting needle to be actuated on the initial image I0 as its initial position, and take the position of the cutting region of the tumor region to be cut on the initial image I0 as its target position, and calculate through the path planning algorithm to obtain the cutting needle movement path of each cutting needle to be actuated.

[0047] On the other hand, the present invention discloses a microscopic automatic cutting device for tumor cell sampling, including:

[0048] A WSI image acquisition module for scanning a glass slide using a digital microscope to obtain a WSI image;

[0049] A tumor region recognition module for using a pre-trained tumor recognition model to identify all tumor regions and their positions on the WSI image;

[0050] An initial image acquisition module for placing one or more glass slides on and fixing them on a stage, and using one or more visual monitoring devices to obtain an initial image I0, on which a section and cutting needles are presented;

[0051] An image registration module, which is used to determine the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image through an image registration algorithm, and obtain the tumor region and its position on the initial image I0;

[0052] A cutting needle recognition module, which is used to recognize the position of the needle tip of the cutting needle on the initial image I0 through a needle tip recognition algorithm;

[0053] A path planning module, which takes the position of the needle tip of the cutting needle on the initial image I0 as the initial position and the position of the tumor region to be cut on the initial image I0 as the target position, and calculates through a path planning algorithm to obtain the movement path of the cutting needle;

[0054] A cutting module, which is used for the cutting needle to drop after moving to the target position according to the obtained movement path of the cutting needle and perform cutting on the corresponding glass slide;

[0055] A cutting monitoring module, which is used to obtain the real-time image I t during the cutting process by using a vision monitoring device, and compare the real-time image I t with the initial image I0 to obtain the deformation amount of the normal tissue region.

[0056] When the deformation amount exceeds the threshold, stop cutting; otherwise, keep cutting until the current tumor region is cut completely;

[0057] A repeated execution module, which is used to move the glass slide after the current tumor region is cut completely, and then repeat the methods in the initial image acquisition module, image registration module, cutting needle recognition module, path planning module, cutting module, and cutting monitoring module until all tumor regions on the glass slide are cut completely to realize tumor cell sampling.

[0058] As a preferred solution, the image registration module includes:

[0059] An image registration unit, which is used to register the initial image I0 and the WSI image through an image registration algorithm with the initial image I0 as the reference image and the WSI image as the floating image;

[0060] A coordinate system corresponding unit, which is used to determine the position of the initial image I0 on the WSI image after image registration, and obtain the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image;

[0061] An initial image tumor region determination unit, which is used to obtain the tumor region and its position on the initial image I0 based on all tumor regions and their positions on the WSI image according to the relative relationship between the two coordinate systems.

[0062] As a preferred solution, the cutting needle recognition module includes:

[0063] A template matching unit, which takes the needle tip image of the cutting needle as a template image, slides pixel by pixel on the initial image I0, and uses a similarity measurement method to calculate and record the matching degree between the template image and the sliding area of the initial image I0 at each position.

[0064] A cutting needle position determination unit, which is used to find the position with the highest matching degree, that is, the position of the cutting needle on the initial image I0.

[0065] As a preferred solution, the cutting monitoring module includes:

[0066] A real-time normal tissue acquisition unit, which is used to obtain a real-time image I during the cutting process by using a vision monitoring device, t and obtain a real-time normal tissue area N according to the following formula t :

[0067] N t = I t - T0;

[0068] where: T0 is the tumor area on the initial image I0;

[0069] An initial normal tissue acquisition unit, which is used to obtain an initial normal tissue area N0 according to the following formula:

[0070] N0 = I0 - T0;

[0071] A deformation amount calculation unit, which is used to calculate the average value of the sum of squares of the differences between the corresponding pixel values of the real-time normal tissue area N t and the initial normal tissue area N0, obtain the deformation amount of the normal tissue area, and compare it with a threshold. When the deformation amount exceeds the threshold, stop cutting;

[0072] A repeated execution unit, which is used to repeat the methods in the real-time normal tissue acquisition unit, the initial normal tissue acquisition unit, and the deformation amount calculation unit until the current tumor area is completely cut.

[0073] As a preferred solution, the path planning module includes:

[0074] A feature extraction unit, which is used to extract features from a part of the tumor area to be cut to obtain tumor features;

[0075] A complexity recognition unit, which is used to judge the complexity of the tumor area to be cut by using a trained tumor complexity judgment model, and output a complexity level T and the number of cutting needles n;

[0076] A cutting needle selection unit, which is used to select the n cutting needles closest to the tumor area to be cut as the cutting needles to be actuated, and allocate corresponding cutting areas to each cutting needle to be actuated;

[0077] A path planning unit, which uses the needle tip position of each cutting needle to be actuated on the initial image I0 as its initial position, and the position of the cutting area of the tumor area to be cut on the initial image I0 as its target position, and calculates through a path planning algorithm to obtain the cutting needle movement path of each cutting needle to be actuated.

[0078] The present invention discloses a microscopic automatic cutting method and device for tumor cell sampling, which has the following beneficial effects:

[0079] First, the present invention can accurately identify the position of the tumor area, and achieve high-precision positioning and fully automatic cutting.

[0080] Second, the present invention associates the WSI image with the image collected by the visual monitoring device through an image registration algorithm, and maps the tumor area on the WSI image to the initial image, ensuring high precision and full automation in the cutting process.

[0081] Third, the present invention uses a visual monitoring device to monitor the cutting process, and based on the deformation amount of the normal tissue area, ensures the stability of the cutting process.

[0082] Fourth, the device adopted by the present invention is inexpensive, easy to operate, has a relatively high cutting throughput, avoids laser thermal damage, and has precise cutting. Description of the Drawings

[0083] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0084] Figure 1 It is a flowchart of the microscopic automatic cutting method provided by the embodiment of the present invention.

[0085] Figure 2 It is one of the structural schematic diagrams of the microtome provided by the embodiment of the present invention.

[0086] Figure 3 It is a schematic diagram of tumor area recognition on the WSI image provided by the embodiment of the present invention.

[0087] Figure 4 It is a schematic diagram of the initial image I0 provided by the embodiment of the present invention.

[0088] Figure 5 It is an effect diagram of image registration provided by the embodiment of the present invention.

[0089] Figure 6Schematic diagram of needle tip recognition of the cutting needle provided by the embodiment of the present invention.

[0090] Figure 7(a) is a schematic diagram of the reciprocating cutting of the cutting needle provided by the embodiment of the present invention;

[0091] Figure 7(b) is a schematic diagram of the same-direction cutting of the cutting needle provided by the embodiment of the present invention.

[0092] Figure 8 It is the second schematic diagram of the structure of the microdissection instrument provided by the embodiment of the present invention.

[0093] Figure 9 It is the third schematic diagram of the structure of the microdissection instrument provided by the embodiment of the present invention.

[0094] Wherein: 1 - visual monitoring device, 2 - cutting needle control device, 3 - algorithm processing device, 4 - supplementary light source, 5 - slide placement platform, 6 - slide, 7 - cutting needle. Detailed implementation manners

[0095] The preferred implementation manners of the present invention will be described in detail below with reference to the accompanying drawings.

[0096] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0097] The expression of "including" elements is an "open" expression. This "open" expression only means the existence of corresponding components or steps, and should not be construed as excluding additional components or steps.

[0098] In order to achieve the purpose of the present invention, in some embodiments of the micro-automatic cutting method for tumor cell sampling, as Figure 1 shown, the micro-automatic cutting method includes:

[0099] Step S1: Scanning the slide using a digital microscope to obtain a WSI image;

[0100] Step S2: Using a pre-trained tumor recognition model to identify all tumor regions and their positions on the WSI image;

[0101] Step S3: Placing and fixing a slide on the stage, and using a visual monitoring device 1 to obtain an initial image I0, on which a section and a cutting needle are presented;

[0102] Step S4: Determine the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image through an image registration algorithm, and obtain the tumor region and its position on the initial image I0;

[0103] Step S5: Identify the position of the cutting needle tip on the initial image I0 through a needle recognition algorithm;

[0104] Step S6: Taking the position of the cutting needle tip on the initial image I0 as the initial position and the position of the tumor region to be cut on the initial image I0 as the target position, calculate through a path planning algorithm to obtain the cutting needle movement path;

[0105] Step S7: After the cutting needle moves to the target position according to the obtained cutting needle movement path and drops, perform cutting on the corresponding glass slide;

[0106] During the cutting process, use the visual monitoring device 1 to obtain the real-time image I t , and compare the real-time image I t with the initial image I0 to obtain the deformation amount of the normal tissue region.

[0107] When the deformation amount exceeds the threshold, stop cutting; otherwise, keep cutting until the current tumor region is cut completely;

[0108] Step S8: After the current tumor region is cut completely, move the glass slide, and then repeat steps S3 - S7 until all tumor regions on the glass slide are cut, realizing tumor cell sampling.

[0109] In the prior art, there are problems such as high equipment cost, high operation requirements, low cutting throughput, risk of thermal damage, and limited cutting accuracy. Therefore, the present invention discloses a microscopic automatic cutting method based on intelligent algorithms, which uses a microdissection instrument for cutting. As Figure 2 shown, the microdissection instrument includes but is not limited to: a sample stage, a visual monitoring device 1, a cutting needle control device 2, an algorithm processing device 3, etc.

[0110] The sample stage includes: a base, a supplementary light source 4, and a glass slide placement platform 5. The glass slide placement platform 5 is used to place and fix the glass slide, and the glass slide placement platform 5 is movable. The supplementary light source 4 can be an LED light source to supplement light for the visual monitoring device 1. The sample stage can move in the X and Y directions to facilitate the replacement and movement of samples. At the same time, in the Z direction, it can be finely adjusted to help the visual monitoring device 1 focus.

[0111] The visual monitoring device 1 is located directly above the sample stage and includes but is not limited to: an industrial camera (such as: 20 million pixels) and a 10x objective lens, for image acquisition and real-time monitoring of the glass slide 6 and the cutting needle 7 on the sample stage.

[0112] The cutting needle control device 2 includes: a power supply, a driving motor, a three-axis motion slider, etc., which controls the cutting needle to move with high precision in the XYZ three-axis directions according to coordinate information for cutting tumor tissues. In some embodiments, a pressure sensor is installed in the Z-axis direction of the cutting needle to prevent the cutting needle from damaging the glass slide. The cutting needle 7 can be but is not limited to a tungsten steel needle with a diameter of 0.3 mm.

[0113] In some other embodiments, the cutting needle 7 is installed on an adapter fixture, and metal magnetic attraction points that magnetically attract each other are provided on both the adapter fixture and the microdissection main body. Before microdissection, the adapter fixture with the cutting needle 7 is brought close to the microdissection main body, and the two are magnetically aligned due to the magnetic force. The metal magnetic attraction point of the adapter fixture contacts the metal magnetic attraction point of the microdissection main body to form a circuit, and the microdissection device can operate.

[0114] The algorithm processing device 3 can be but is not limited to including: a workstation such as an Intel Core i9 processor, 32 GB of memory, and an NVIDIA RTX 3090 graphics card. The algorithm processing device 3 has a variety of algorithms built in.

[0115] Each of the above steps will be described in detail below.

[0116] Step S1: Use a digital microscope to scan the glass slide to obtain a whole slide image, i.e., a WSI image.

[0117] Step S2: Use a pre-trained tumor recognition model to identify all tumor regions and their positions on the WSI image.

[0118] As Figure 3 shown, the tumor recognition model will identify all tumor cells on the WSI image, and the regions where these tumor cells are located are recorded as tumor regions ) . Since artificial intelligence tumor recognition based on WSI is a mature technology, it will not be elaborated here.

[0119] Step S3: Place and fix a glass slide on the stage, drop an appropriate amount of paraffin oil on the sample, finely adjust the position of the stage, and through the XY directions, make the sample in the visual acquisition area of the visual monitoring device. Through the fine adjustment of the Z axis, make the visual monitoring device able to focus and collect a clear initial image I0. The initial image I0 shows a section and a cutting needle, as Figure 4 shown.

[0120] Step S4: Through an image registration algorithm, determine the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image, and obtain the tumor regions and their positions on the initial image I0.

[0121] Specifically, step S4 includes:

[0122] Step S4.1: Using the initial image I0 as the reference image and the WSI image as the floating image, register the initial image I0 with the WSI image through an image registration algorithm (such as: a registration algorithm based on the sift feature operator), as Figure 5 shown;

[0123] Step S4.2: After image registration, determine the position of the initial image I0 on the WSI image, and obtain the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image;

[0124] Step S4.3: According to the relative relationship between the two coordinate systems, based on all tumor regions and their positions on the WSI image, obtain the tumor regions and their positions on the initial image I0, denoted as T0.

[0125] The above image registration algorithm belongs to a mature technology, so it will not be elaborated here.

[0126] Step S5 uses a needle recognition algorithm to identify the position of the cutting needle tip on the initial image I0. The needle recognition algorithm can adopt the method of template matching.

[0127] Specifically, Step S5 includes:

[0128] Step S5.1: Using the image of the cutting needle tip as the template image, slide it pixel by pixel on the initial image I0, and use a similarity measurement method to calculate and record the matching degree between the template image and the sliding area of the initial image I0 at each position;

[0129] Step S5.2: Find the position with the highest matching degree, which is the position P0 of the cutting needle on the initial image I0, as Figure 6 shown.

[0130] Step S6 takes the position of the cutting needle tip on the initial image I0 as the initial position and the position of the tumor region to be cut on the initial image I0 as the target position, and calculates through a path planning algorithm to obtain the movement path of the cutting needle.

[0131] Step S7 takes the current needle tip position as the zero position, raises the cutting needle, moves to the target position according to the obtained movement path of the cutting needle and then drops, and cuts on the corresponding glass slide with a cutting depth of 3 - 5 microns.

[0132] Within the tumor region T0, the path of the cutting needle is scraped row by row, such as: back-and-forth scraping and unidirectional scraping. As shown in Figure 7(a), the back-and-forth scraping has higher efficiency, but may be affected by the gap of the control lead screw; as shown in Figure 7(b), the unidirectional scraping has lower efficiency, but can effectively avoid the error caused by the lead screw gap.

[0133] During the cutting process, use a visual monitoring device to obtain the real-time image It and compare the real-time image I t with the initial image I0 to obtain the deformation amount of the normal tissue area. When the deformation amount exceeds the threshold, stop cutting; otherwise, keep cutting until the tumor area is completely cut.

[0134] Specifically, step S7 includes:

[0135] Step S7.1: During the cutting process, use a visual monitoring device to obtain the real-time image I t , and obtain the real-time normal tissue area N t according to the following formula:

[0136] N t = I t - T0;

[0137] where: T0 is the tumor area on the initial image I0;

[0138] Step S7.2: Obtain the initial normal tissue area N0 according to the following formula:

[0139] N0 = I0 - T0;

[0140] Step S7.3: Calculate the average value of the sum of squares of the differences between the corresponding pixel values of the real-time normal tissue area N t and the initial normal tissue area N0 to obtain the deformation amount of the normal tissue area, and compare it with the threshold. When the deformation amount exceeds the threshold, stop cutting;

[0141] Step S7.4: Repeat steps S7.1 - S7.3 until the current tumor area is completely cut.

[0142] The present invention calculates the mean square error (MSE) between the normal tissue area of the real-time image and the normal tissue area of the initial image to understand the deformation of the image. If the deformation is normal, continue cutting. If the difference is large, interrupt cutting to prevent further tissue damage.

[0143] After the current tumor area is completely cut in step S8, move the glass slide to the next area by moving the loading platform, and then repeat steps S3 - S7 until all tumor areas on the glass slide are cut to achieve tumor cell sampling.

[0144] Collect the cut tumor cell tissue for molecular detection.

[0145] In some other embodiments, as Figure 8 shown, multiple cutting needles can be used in cooperation for cutting.

[0146] Specifically, step S6 includes:

[0147] Step S6.1: Extract features from a part of the tumor area to be cut to obtain tumor features, including: morphological features (such as area, perimeter, compactness, irregularity of the tumor boundary, etc.), texture features (such as microscopic texture features, etc.), and heterogeneity features (such as multi-region color differences, uneven nuclear distribution, etc.);

[0148] Step S6.2: Use the trained tumor complexity judgment model to judge the complexity of the tumor area to be cut, and output the complexity level T and the number of cutting needles n;

[0149] For example, for low complexity, use 1 cutting needle;

[0150] For medium complexity, use 2 cutting needles;

[0151] For high complexity, use 3 cutting needles;

[0152] Step S6.3: Select the n cutting needles closest to the tumor area to be cut as the cutting needles to be actuated, and allocate corresponding cutting areas to each cutting needle to be actuated;

[0153] Step S6.4: Take the needle tip position of each cutting needle to be actuated on the initial image I0 as its initial position, and take the position of the cutting area of the tumor area to be cut on the initial image I0 as its target position, and calculate through the path planning algorithm to obtain the cutting needle movement path of each cutting needle to be actuated.

[0154] Of course, in some other embodiments, multiple cutting needles can cut multiple tumor areas simultaneously, adopting a one-to-one allocation method to improve the cutting efficiency. Multiple cutting needles all preferentially cut the tumor areas close to themselves to prevent interference with each other during the cutting process.

[0155] In some other embodiments, as Figure 9 shown, to improve the degree of automation, the present invention can cut multiple glass slides at one time and can adopt multiple visual monitoring devices.

[0156] Same as the above embodiment, the difference lies in the above step S3.

[0157] Step S3 is to place and fix multiple glass slides on the loading platform, use multiple visual monitoring devices to obtain images, splice the obtained images to obtain the initial image I0, and the initial image I0 presents sections and cutting needles.

[0158] The present invention discloses a microscopic automatic cutting method for tumor cell sampling, which has the following beneficial effects:

[0159] First, the equipment cost is low. There is no need for expensive laser light sources and high-precision microscopes, only ordinary cameras and cutting needles are required, which greatly reduces the equipment investment cost and is conducive to popularization and application in more laboratories and research institutions.

[0160] Second, the operation is simple. There is no need for long-term professional training. Just place the glass slide on the stage, and it can automatically identify tumor cells and plan the cutting path. The operator does not need to be proficient in complex laser parameter settings and sample processing procedures.

[0161] Third, the cutting throughput is high. It can automatically cut the whole section, greatly improving the cutting efficiency and meeting the needs of large-scale and high-throughput sample collection.

[0162] Fourth, laser thermal damage is avoided. There is no need for laser irradiation during the cutting process, which will not cause thermal damage to the sample, ensuring the activity of biomacromolecules in the cells and providing reliable materials for subsequent molecular biology detection.

[0163] Fifth, the cutting accuracy is high. With the help of artificial intelligence algorithms, it can accurately identify tumor cells on the section and achieve precise cutting through path planning algorithms, minimizing the contamination of surrounding irrelevant cells to the greatest extent, ensuring the sample purity, and making the results of the extracted samples closer to the in-vivo real state when performing macromolecule detections such as nucleic acids and proteins.

[0164] In some other embodiments, the present invention discloses a microscopic automatic cutting device for tumor cell sampling, including:

[0165] A WSI image acquisition module for scanning a glass slide using a digital microscope to obtain a WSI image;

[0166] A tumor region recognition module for using a pre-trained tumor recognition model to identify all tumor regions and their positions on the WSI image;

[0167] An initial image acquisition module for placing and fixing one or more glass slides on the stage and using one or more visual monitoring devices to obtain an initial image I0, on which a section and a cutting needle are presented;

[0168] An image registration module for determining the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image through an image registration algorithm, and obtaining the tumor regions and their positions on the initial image I0;

[0169] A cutting needle recognition module for identifying the position of the cutting needle tip on the initial image I0 through a needle tip recognition algorithm;

[0170] A path planning module, which uses the needle tip position of the cutting needle on the initial image I0 as the initial position and the position of the tumor area to be cut on the initial image I0 as the target position, and calculates through a path planning algorithm to obtain the movement path of the cutting needle;

[0171] A cutting module, which is used for the cutting needle to drop after moving to the target position according to the obtained movement path of the cutting needle and perform cutting on the corresponding glass slide;

[0172] A cutting monitoring module, which is used to obtain a real-time image I t during the cutting process using a vision monitoring device, t compare the real-time image I

[0173] with the initial image I0 to obtain the deformation amount of the normal tissue area. When the deformation amount exceeds the threshold, stop cutting; otherwise, keep cutting until the current tumor area is cut;

[0174] A repeated execution module, which is used to move the glass slide after the current tumor area is cut, and then repeat the methods in the initial image acquisition module, image registration module, cutting needle recognition module, path planning module, cutting module, and cutting monitoring module until all tumor areas on the glass slide are cut to achieve tumor cell sampling.

[0175] Furthermore, the image registration module includes:

[0176] An image registration unit, which uses the initial image I0 as the reference image and the WSI image as the floating image, and registers the initial image I0 and the WSI image through an image registration algorithm;

[0177] A coordinate system corresponding unit, which is used to determine the positions of the initial image I0 on the WSI image after image registration, and obtain the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image;

[0178] An initial image tumor area determination unit, which is used to obtain the tumor area and its position on the initial image I0 based on all tumor areas and their positions on the WSI image according to the relative relationship between the two coordinate systems.

[0179] As a preferred solution, the cutting needle recognition module includes:

[0180] A template matching unit, which uses the needle tip image of the cutting needle as the template image, slides pixel by pixel on the initial image I0, and calculates and records the matching degree between the template image and the sliding area of the initial image I0 at each position using a similarity measurement method;

[0181] A cutting needle position determination unit, which is used to find the position with the highest matching degree, which is the position of the cutting needle on the initial image I0.

[0182] Furthermore, the cutting monitoring module includes:

[0183] A real-time normal tissue acquisition unit, which is used to obtain a real-time image I during the cutting process by using a visual monitoring device t , and obtain a real-time normal tissue region N according to the following formula t :

[0184] N t = I t - T0;

[0185] Where: T0 is the tumor region on the initial image I0;

[0186] An initial normal tissue acquisition unit, which is used to obtain an initial normal tissue region N0 according to the following formula:

[0187] N0 = I0 - T0;

[0188] A deformation amount calculation unit, which is used to calculate the average value of the sum of squares of the differences between the corresponding pixel values of the real-time normal tissue region N t and the initial normal tissue region N0, obtain the deformation amount of the normal tissue region, and compare it with a threshold. When the deformation amount exceeds the threshold, stop cutting;

[0189] A repeated execution unit, which is used to repeat the methods in the real-time normal tissue acquisition unit, the initial normal tissue acquisition unit, and the deformation amount calculation unit until the current tumor region is completely cut.

[0190] Furthermore, the path planning module includes:

[0191] A feature extraction unit, which is used to extract features from a part of the tumor region to be cut and obtain tumor features;

[0192] A complexity identification unit, which is used to judge the complexity of the tumor region to be cut by using a trained tumor complexity judgment model and output a complexity level T and the number of cutting needles n;

[0193] A cutting needle selection unit, which is used to select the n cutting needles closest to the tumor region to be cut as the cutting needles to be actuated, and allocate corresponding cutting regions to each cutting needle to be actuated;

[0194] A path planning unit, which is used to use the needle tip positions of each cutting needle to be actuated on the initial image I0 as their initial positions, and use the positions of the cutting regions of the tumor region to be cut on the initial image I0 as their target positions, and calculate through a path planning algorithm to obtain the cutting needle movement paths of each cutting needle to be actuated.

[0195] Furthermore, it should be noted that when the micro-automatic cutting device for tumor cell sampling provided in the above embodiments cuts tumor cells, only the division of the above functional modules is used as an example for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the micro-automatic cutting device for tumor cell sampling is divided into different functional modules to complete all or part of the functions described above.

[0196] In addition, the micro-automatic cutting device for tumor cell sampling provided in the above embodiments and the embodiments of the micro-automatic cutting method for tumor cell sampling belong to the same concept. For the specific implementation process, please refer to the method embodiments and will not be elaborated here.

[0197] The present invention discloses a micro-automatic cutting method and device for tumor cell sampling, which has the following beneficial effects:

[0198] First, the present invention can accurately identify the location of the tumor area, realizing high-precision positioning and fully automatic cutting.

[0199] Second, the present invention associates the WSI image with the image collected by the visual monitoring device through an image registration algorithm, and maps the tumor area on the WSI image to the initial image, ensuring high precision and full automation in the cutting process.

[0200] Third, the present invention uses a visual monitoring device to monitor the cutting process and ensures the stability of the cutting process based on the deformation amount of the normal tissue area.

[0201] Fourth, the device adopted by the present invention is inexpensive, easy to operate, has a relatively high cutting throughput, avoids laser thermal damage, and has precise cutting.

[0202] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of the present invention claimed is defined by the appended claims and their equivalents.

Claims

1. A microscopic automatic cutting method for tumor cell sampling, characterized in that: include: Step S1: Scan the slide using a digital microscope to obtain a WSI image; Step S2: using the pre-trained tumor recognition model to identify all tumor areas and their locations on the WSI image; Step S3: placing and fixing one or more glass slides on a loading platform, and using one or more visual monitoring devices to obtain an initial image I0, where a slice and a cutting needle are shown on the initial image I0; Step S4: determining the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image through an image registration algorithm, and obtaining the tumor area and its position on the initial image I0; Step S5: using a needle recognition algorithm, identifying the position of the cutting needle tip on the initial image I0; Step S6: Taking the needle tip position of the cutting needle on the initial image I0 as the initial position and the position of the tumor area to be cut on the initial image I0 as the target position, the movement path of the cutting needle is obtained by calculation through a path planning algorithm; Step S7: the cutting needle moves to the target position according to the obtained cutting needle movement path and then falls down to perform cutting on the corresponding glass slide; During the cutting process, a visual monitoring device is used to obtain real-time images. t , and the real-time image I t Compared with the initial image I0, the deformation of the normal tissue area is obtained. When the deformation exceeds the threshold, the cutting is stopped; otherwise, the cutting is continued until the current tumor area is cut completely; Step S8: After the cutting of the current tumor area is completed, the slide is moved and steps S3 to S7 are repeated until all tumor areas on the slide are cut to achieve tumor cell sampling.

2. The micro-automatic dissection method according to claim 1, characterized in that: The step S4 comprises: Step S4.1: using the initial image I0 as a reference image and the WSI image as a floating image, the initial image I0 and the WSI image are registered by an image registration algorithm; Step S4.2: After image registration, determine the position of the initial image I0 on the WSI image, and obtain the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image; Step S4.3: According to the relative relationship between the two coordinate systems, based on all tumor regions and their positions on the WSI image, the tumor regions and their positions on the initial image I0 are obtained.

3. The micro-automatic dissection method according to claim 1, characterized in that: The step S5 comprises: Step S5.1: Using the needle tip image of the cutting needle as a template image, sliding pixel by pixel on the initial image I0, and using a similarity measurement method to calculate and record the matching degree between the template image and the sliding area of ​​the initial image I0 at each position; Step S5.2: Find the position with the highest matching degree, which is the position of the cutting needle in the initial image I0.

4. The micro-automatic dissection method according to claim 1, characterized in that: The step S7 comprises: Step S7.1: During the cutting process, use a visual monitoring device to obtain real-time images I t , the real-time normal tissue area N is obtained according to the following formula t : N t = I t -T0; Where: T0 is the tumor area on the initial image I0; Step S7.2: Obtain the initial normal tissue area N0 according to the following formula: N0= I0- T0; Step S7.3: Calculate the real-time normal tissue area N t The average value of the sum of squares of the differences between the initial normal tissue area N0 and the corresponding pixel values ​​is used to obtain the deformation of the normal tissue area, and the deformation is compared with the threshold. When the deformation exceeds the threshold, the cutting is stopped. Step S7.4: Repeat steps S7.1 to S7.3 until the current tumor area is cut.

5. The micro-automatic dissection method according to claim 1, characterized in that: The step S6 comprises: Step S6.1: extracting features from the part of the tumor area to be cut to obtain tumor features; Step S6.2: Use the trained tumor complexity judgment model to judge the complexity of the tumor area to be cut, and output the complexity level T and the number of cutting needles n; Step S6.3: selecting n cutting needles closest to the tumor area to be cut as waiting cutting needles, and assigning a corresponding cutting area to each waiting cutting needle; Step S6.4: Taking the needle tip position of each cutting needle to be activated on the initial image I0 as its initial position, and taking the position of the cutting area of ​​the tumor area to be cut on the initial image I0 as its target position, the cutting needle motion path of each cutting needle to be activated is obtained through calculation by the path planning algorithm.

6. A microscopic automatic cutting device for tumor cell sampling, characterized in that: include: A WSI image acquisition module, used to scan the slide using a digital microscope to acquire a WSI image; The tumor region recognition module is used to identify all tumor regions and their locations on the WSI image using a pre-trained tumor recognition model; An initial image acquisition module is used to place and fix one or more glass slides on the loading platform, and use one or more visual monitoring devices to obtain an initial image I0, on which a slice and a cutting needle are presented; An image registration module, used to determine the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image through an image registration algorithm, and obtain the tumor area and its position on the initial image I0; A cutting needle recognition module is used to identify the position of the cutting needle tip on the initial image I0 through a needle tip recognition algorithm; A path planning module is used to obtain a movement path of the cutting needle by using the needle tip position of the cutting needle on the initial image I0 as the initial position and the position of the tumor area to be cut on the initial image I0 as the target position through calculation by a path planning algorithm; A cutting module, used for the cutting needle to move to a target position according to the obtained cutting needle movement path and then fall down to perform cutting on the corresponding glass slide; Cutting monitoring module, used to obtain real-time images using visual monitoring devices during the cutting process t , and the real-time image I t Compared with the initial image I0, the deformation of the normal tissue area is obtained. When the deformation exceeds the threshold, the cutting is stopped; otherwise, the cutting is continued until the current tumor area is cut completely; The repeated execution module is used to move the glass slide after the current tumor area is cut, and then repeat the methods in the initial image acquisition module, image registration module, cutting needle recognition module, path planning module, cutting module, and cutting monitoring module until all tumor areas on the glass slide are cut to achieve tumor cell sampling.

7. The microautomatic dissection device according to claim 6, characterized in that: The image registration module comprises: An image registration unit, used for registering the initial image I0 with the WSI image by using an image registration algorithm, taking the initial image I0 as a reference image and the WSI image as a floating image; A coordinate system corresponding unit, used to determine the position of the initial image I0 on the WSI image after image registration, and obtain the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image; The initial image tumor region determining unit is used to obtain the tumor region and its position on the initial image I0 based on the relative relationship between the two coordinate systems and all the tumor regions and their positions on the WSI image.

8. The microautomatic dissection device according to claim 6, characterized in that: The cutting needle identification module comprises: The template matching unit is used to use the needle head image of the cutting needle as a template image, slide pixel by pixel on the initial image I0, and calculate and record the matching degree between the template image and the sliding area of ​​the initial image I0 at each position by using a similarity measurement method; The cutting needle position determination unit is used to find the position with the highest matching degree, that is, the position of the cutting needle in the initial image I0.

9. The microautomatic dissection device according to claim 6, characterized in that: The cutting monitoring module comprises: Real-time normal tissue acquisition unit, used to obtain real-time images using a visual monitoring device during the cutting process t , the real-time normal tissue area N is obtained according to the following formula t : N t = I t -T0; Where: T0 is the tumor area on the initial image I0; The initial normal tissue acquisition unit is used to obtain the initial normal tissue area N0 according to the following formula: N0= I0- T0; Deformation calculation unit, used to calculate the real-time normal tissue area N t The average value of the sum of squares of the differences between the initial normal tissue area N0 and the corresponding pixel values ​​is used to obtain the deformation of the normal tissue area, and the deformation is compared with the threshold. When the deformation exceeds the threshold, the cutting is stopped. The repetitive execution unit is used to repeat the methods in the real-time normal tissue acquisition unit, the initial normal tissue acquisition unit, and the deformation calculation unit until the current tumor area is cut.

10. The microautomatic dissection device according to claim 6, characterized in that: The path planning module includes: A feature extraction unit, used for extracting features from a portion of the tumor area to be cut to obtain tumor features; The complexity recognition unit is used to judge the complexity of the tumor area to be cut using the trained tumor complexity judgment model, and output the complexity level T and the number of cutting needles n; A cutting needle selection unit is used to select n cutting needles closest to the tumor area to be cut as waiting cutting needles, and assign a corresponding cutting area to each waiting cutting needle; The path planning unit is used to obtain the cutting needle motion path of each cutting needle to be activated by using the needle head position of each cutting needle to be activated on the initial image I0 as its initial position and the position of the cutting area of ​​the tumor area to be cut on the initial image I0 as its target position, and calculating through the path planning algorithm.

Images