Microscopic automatic cutting method and device for tumor cell sampling

Abstract

The invention discloses a microscopic automatic cutting method and device for tumor cell sampling. The method comprises the following steps: acquiring a WSI image; identifying all tumor areas and positions thereof on the WSI image; acquiring an initial image by adopting a visual monitoring device; obtaining a tumor area and a position thereof on the initial image; identifying the position of the needle head of the cutting needle on the initial image; calculating through a path planning algorithm to obtain a motion path of the cutting needle; cutting on the slide; in the cutting process, a visual monitoring device is used for obtaining a real-time image, the real-time image is compared with the initial image, the deformation quantity of the normal tissue area is obtained, and when the deformation quantity exceeds a threshold value, cutting is stopped; otherwise, performing cutting until the current tumor area is cut completely; and after the current tumor area is cut, moving the slide, and repeating the steps until all tumor areas on the slide are cut. According to the invention, the position of the tumor area can be accurately identified, and high-precision positioning and full-automatic cutting are realized.

Description

Technical Field

[0001] The invention belongs to the field of biomedical technology, and in particular relates to a microscopic automatic cutting method and equipment for tumor cell sampling. Background Art

[0002] Isolating the desired cell population from tissue sections or cell smears is essential for subsequent molecular biological analysis. Currently, laser capture microdissection and mechanical microdissection are two commonly used cell isolation methods.

[0003] Laser capture microdissection technology uses a laser beam to shoot vertically from above the slice, and the light spot falls on the area to be cut. When the film in the area is lifted, the cells connected to it are also cut off. This technology has the advantages of precise cutting, flexible operation, good sample integrity, and can be used in conjunction with a variety of analytical techniques. However, it also has disadvantages such as high equipment cost, high requirements for operators, relatively low cutting flux, and the risk of thermal damage.

[0004] Mechanical microdissection technology uses special micro-knives, blades or cutting needles to directly cut tissues or cells under a microscope. This technology is simple and intuitive, relatively low-cost and easy to understand. However, the cutting process is manually operated, and the cutting accuracy is affected by individual differences between operators. It is difficult to handle tiny, delicate and dispersed cell groups, and the cutting efficiency is low.

[0005] In addition, whether it is laser capture microdissection or mechanical microdissection, there is a balance between field of view and efficiency. If a higher magnification is used, the display field of view will be reduced and the efficiency of the operation will be low. If a lower magnification is used, although the field of view is expanded, the magnification of the image is too low, which will affect the accuracy of observation.

[0006] The existing laser capture microdissection technology has the following problems: First, the existing laser capture microdissection technology equipment is expensive, including the instrument itself, laser light source, high-precision microscope, professional control software, etc., which all require high investment, and the subsequent maintenance costs are also high, which limits its popularization and application in some laboratories or scientific research institutions with limited budgets.

[0007] Second, laser capture microdissection technology has high requirements for operators, who need to be proficient in microscope operation skills, laser parameter settings, and sample processing procedures. It requires professional and long-term training to get started, and the operation requirements are high.

[0008] Third, the cutting throughput of laser capture microdissection technology is relatively low, and it can only process samples in a small area at a time. It is inefficient for large-scale, high-throughput sample collection tasks.

[0009] Fourth, laser capture microdissection technology has a certain risk of thermal damage. When used for a long time and with high intensity, it may cause thermal damage to the sample, affect the activity of intracellular biomacromolecules, and interfere with subsequent molecular biology detection results.

[0010] Fifth, the cutting process of mechanical microdissection technology is manually operated, and the cutting accuracy is affected by individual differences of operators. It is difficult to handle tiny, delicate and dispersed cell groups, and the cutting efficiency is low.

[0011] Therefore, there is an urgent need for a new type of microdissection technology that can reduce equipment costs and improve the degree of automation while maintaining the advantages of high-precision cutting, flexible operation, and good sample integrity, so as to meet the needs of biomedical research for high-quality biological samples. Summary of the invention

[0012] In order to solve the above technical problems, the present invention proposes a microscopic automatic cutting method and equipment for tumor cell sampling.

[0013] In order to achieve the above object, the technical solution of the present invention is as follows: In one aspect, the present invention discloses a microscopic automatic cutting method for tumor cell sampling, comprising: Step S1: Scan the slide using a digital microscope to obtain a WSI image; Step S2: using the pre-trained tumor recognition model to identify all tumor areas and their locations on the WSI image; Step S3: placing and fixing one or more glass slides on a loading platform, and using one or more visual monitoring devices to obtain an initial image I0, where a slice and a cutting needle are shown on the initial image I0; Step S4: determining the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image through an image registration algorithm, and obtaining the tumor area and its position on the initial image I0; Step S5: using a needle recognition algorithm, identifying the position of the cutting needle tip on the initial image I0; Step S6: Taking the needle tip position of the cutting needle on the initial image I0 as the initial position and the position of the tumor area to be cut on the initial image I0 as the target position, the movement path of the cutting needle is obtained by calculation through a path planning algorithm; Step S7: the cutting needle moves to the target position according to the obtained cutting needle movement path and then falls down to perform cutting on the corresponding glass slide; During the cutting process, a visual monitoring device is used to obtain real-time images. t , and the real-time image I t Compared with the initial image I0, the deformation of the normal tissue area is obtained. When the deformation exceeds the threshold, the cutting is stopped; otherwise, the cutting is continued until the current tumor area is cut completely; Step S8: After the cutting of the current tumor area is completed, the slide is moved and steps S3 to S7 are repeated until all tumor areas on the slide are cut to achieve tumor cell sampling.

[0014] Based on the above technical solution, the following improvements can be made: As a preferred solution, step S4 includes: Step S4.1: using the initial image I0 as a reference image and the WSI image as a floating image, the initial image I0 and the WSI image are registered by an image registration algorithm; Step S4.2: After image registration, determine the position of the initial image I0 on the WSI image, and obtain the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image; Step S4.3: According to the relative relationship between the two coordinate systems, based on all tumor regions and their positions on the WSI image, the tumor regions and their positions on the initial image I0 are obtained.

[0015] As a preferred solution, step S5 includes: Step S5.1: Using the needle tip image of the cutting needle as a template image, sliding pixel by pixel on the initial image I0, and using a similarity measurement method to calculate and record the matching degree between the template image and the sliding area of ​​the initial image I0 at each position; Step S5.2: Find the position with the highest matching degree, which is the position of the cutting needle in the initial image I0.

[0016] As a preferred solution, step S7 includes: Step S7.1: During the cutting process, use a visual monitoring device to obtain real-time images I t , the real-time normal tissue area N is obtained according to the following formula t : N t = I t - T0; Where: T0 is the tumor area on the initial image I0; Step S7.2: Obtain the initial normal tissue area N0 according to the following formula: N0= I0- T0; Step S7.3: Calculate the real-time normal tissue area N t The average value of the sum of squares of the differences between the initial normal tissue area N0 and the corresponding pixel values ​​is used to obtain the deformation of the normal tissue area, and the deformation is compared with the threshold. When the deformation exceeds the threshold, the cutting is stopped. Step S7.4: Repeat steps S7.1 to S7.3 until the current tumor area is cut.

[0017] As a preferred solution, step S6 includes: Step S6.1: extracting features from the part of the tumor area to be cut to obtain tumor features; Step S6.2: using the trained tumor complexity judgment model to judge the complexity of the tumor area to be cut, and outputting the complexity level T and the number of cutting needles n; Step S6.3: selecting n cutting needles closest to the tumor area to be cut as waiting cutting needles, and assigning a corresponding cutting area to each waiting cutting needle; Step S6.4: Taking the needle tip position of each cutting needle to be activated on the initial image I0 as its initial position, and taking the position of the cutting area of ​​the tumor area to be cut on the initial image I0 as its target position, the cutting needle motion path of each cutting needle to be activated is obtained through calculation by the path planning algorithm.

[0018] In another aspect, the present invention discloses a microscopic automatic cutting device for tumor cell sampling, comprising: A WSI image acquisition module, used to scan the slide using a digital microscope to acquire a WSI image; The tumor region recognition module is used to identify all tumor regions and their locations on the WSI image using a pre-trained tumor recognition model; An initial image acquisition module is used to place and fix one or more glass slides on the loading platform, and use one or more visual monitoring devices to obtain an initial image I0, on which a slice and a cutting needle are presented; An image registration module, used to determine the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image through an image registration algorithm, and obtain the tumor area and its position on the initial image I0; A cutting needle recognition module is used to identify the position of the cutting needle tip on the initial image I0 through a needle tip recognition algorithm; A path planning module is used to obtain a movement path of the cutting needle by using the needle tip position of the cutting needle on the initial image I0 as the initial position and the position of the tumor area to be cut on the initial image I0 as the target position through calculation by a path planning algorithm; A cutting module, used for the cutting needle to move to a target position according to the obtained cutting needle movement path and then fall down to perform cutting on the corresponding glass slide; Cutting monitoring module, used to obtain real-time images using visual monitoring devices during the cutting process t , and the real-time image I t Compared with the initial image I0, the deformation of the normal tissue area is obtained. When the deformation exceeds the threshold, the cutting is stopped; otherwise, the cutting is continued until the current tumor area is cut completely; The repeated execution module is used to move the glass slide after the current tumor area is cut, and then repeat the methods in the initial image acquisition module, image registration module, cutting needle recognition module, path planning module, cutting module, and cutting monitoring module until all tumor areas on the glass slide are cut to achieve tumor cell sampling.

[0019] As a preferred solution, the image registration module includes: An image registration unit, used for registering the initial image I0 with the WSI image by using an image registration algorithm, taking the initial image I0 as a reference image and the WSI image as a floating image; A coordinate system corresponding unit, used to determine the position of the initial image I0 on the WSI image after image registration, and obtain the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image; The initial image tumor region determining unit is used to obtain the tumor region and its position on the initial image I0 based on the relative relationship between the two coordinate systems and all the tumor regions and their positions on the WSI image.

[0020] As a preferred solution, the cutting needle identification module includes: A template matching unit is used to use the needle head image of the cutting needle as a template image, slide pixel by pixel on the initial image I0, and use a similarity measurement method to calculate and record the matching degree between the template image and the sliding area of ​​the initial image I0 at each position; The cutting needle position determination unit is used to find the position with the highest matching degree, that is, the position of the cutting needle in the initial image I0.

[0021] As a preferred solution, the cutting monitoring module includes: Real-time normal tissue acquisition unit, used to obtain real-time images using a visual monitoring device during the cutting process t , the real-time normal tissue area N is obtained according to the following formula t : N t = I t - T0; Where: T0 is the tumor area on the initial image I0; The initial normal tissue acquisition unit is used to obtain the initial normal tissue area N0 according to the following formula: N0= I0- T0; Deformation calculation unit, used to calculate the real-time normal tissue area N tThe average value of the sum of squares of the differences between the initial normal tissue area N0 and the corresponding pixel values ​​is used to obtain the deformation of the normal tissue area, and the deformation is compared with the threshold. When the deformation exceeds the threshold, the cutting is stopped. The repetitive execution unit is used to repeat the methods in the real-time normal tissue acquisition unit, the initial normal tissue acquisition unit, and the deformation calculation unit until the current tumor area is cut.

[0022] As a preferred solution, the path planning module includes: A feature extraction unit, used for extracting features from a portion of the tumor area to be cut to obtain tumor features; The complexity recognition unit is used to judge the complexity of the tumor area to be cut using the trained tumor complexity judgment model, and output the complexity level T and the number of cutting needles n; A cutting needle selection unit is used to select n cutting needles closest to the tumor area to be cut as waiting cutting needles, and assign a corresponding cutting area to each waiting cutting needle; The path planning unit is used to obtain the cutting needle motion path of each cutting needle to be activated by using the needle head position of each cutting needle to be activated on the initial image I0 as its initial position and the position of the cutting area of ​​the tumor area to be cut on the initial image I0 as its target position, and calculating through the path planning algorithm.

[0023] The present invention discloses a microscopic automatic cutting method and device for tumor cell sampling, which has the following beneficial effects: First, the present invention can accurately identify the location of the tumor area and achieve high-precision positioning and fully automatic cutting.

[0024] Second, the present invention associates the WSI image with the image collected by the visual monitoring device through an image registration algorithm, and maps the tumor area on the WSI image to the initial image, thereby ensuring high precision and full automation of the cutting process.

[0025] Third, the present invention uses a visual monitoring device to monitor the cutting process, based on the deformation of the normal tissue area, to ensure the stability of the cutting process.

[0026] Fourth, the equipment used in the present invention is low-cost, easy to operate, has a high cutting flux, avoids laser thermal damage, and has precise cutting. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 The present invention provides a flowchart of the automatic microdissection method.

[0029] Figure 2 This is one of the structural schematic diagrams of the microdissection instrument provided in an embodiment of the present invention.

[0030] Figure 3 A schematic diagram of tumor region recognition on a WSI image provided by an embodiment of the present invention.

[0031] Figure 4 A schematic diagram of an initial image I0 provided in an embodiment of the present invention.

[0032] Figure 5 This is a diagram showing the effect of image registration provided by an embodiment of the present invention.

[0033] Figure 6 A schematic diagram of needle tip identification of a cutting needle provided in an embodiment of the present invention.

[0034] FIG7 (a) is a schematic diagram of the back-and-forth cutting of the cutting needle provided in an embodiment of the present invention; FIG. 7( b ) is a schematic diagram of the same-direction cutting of the cutting needle provided in an embodiment of the present invention.

[0035] Figure 8 The second structural schematic diagram of the microdissection instrument provided in the embodiment of the present invention.

[0036] Fig. 9 The third structural schematic diagram of the microdissection instrument provided in the embodiment of the present invention.

[0037] Among them: 1-visual monitoring device, 2-cutting needle control device, 3-algorithm processing device, 4-filling light source, 5-glass slide placement platform, 6-glass slide, 7-cutting needle. DETAILED DESCRIPTION

[0038] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The expression of “comprising” an element is an “open” expression, which merely means that a corresponding component or step exists, and should not be interpreted as excluding additional components or steps.

[0041] In order to achieve the purpose of the present invention, some embodiments of the microscopic automatic cutting method for tumor cell sampling, such as Figure 1 As shown, the micro-automatic cutting method comprises: Step S1: Scan the slide using a digital microscope to obtain a WSI image; Step S2: using the pre-trained tumor recognition model to identify all tumor areas and their locations on the WSI image; Step S3: placing and fixing a glass slide on a loading platform, and using a visual monitoring device 1 to obtain an initial image I0, where a slice and a cutting needle are shown; Step S4: determining the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image through an image registration algorithm, and obtaining the tumor area and its position on the initial image I0; Step S5: using a needle recognition algorithm, identifying the position of the cutting needle tip on the initial image I0; Step S6: Taking the needle tip position of the cutting needle on the initial image I0 as the initial position and the position of the tumor area to be cut on the initial image I0 as the target position, the movement path of the cutting needle is obtained by calculation through a path planning algorithm; Step S7: the cutting needle moves to the target position according to the obtained cutting needle movement path and then falls down to perform cutting on the corresponding glass slide; During the cutting process, a visual monitoring device 1 is used to obtain a real-time image I t , and the real-time image I t Compared with the initial image I0, the deformation of the normal tissue area is obtained. When the deformation exceeds the threshold, the cutting is stopped; otherwise, the cutting is continued until the current tumor area is cut completely; Step S8: After the cutting of the current tumor area is completed, the slide is moved and steps S3 to S7 are repeated until all tumor areas on the slide are cut to achieve tumor cell sampling.

[0042] The prior art has problems such as high equipment cost, high operating requirements, low cutting flux, risk of thermal damage and limited cutting accuracy. Therefore, the present invention discloses a micro-automatic cutting method based on an intelligent algorithm, which uses a micro-cutting instrument for cutting. Figure 2 As shown, the microdissection instrument includes but is not limited to: a loading platform, a visual monitoring device 1, a cutting needle control device 2, an algorithm processing device 3, etc.

[0043] The loading platform includes: a base, a fill light source 4 and a slide placement platform 5. The slide placement platform 5 is used to place and fix the slide, and the slide placement platform 5 is movable. The fill light source 4 can be an LED light source to fill light for the visual monitoring device 1. The loading platform can be moved in the X and Y directions to facilitate the replacement and movement of the sample. At the same time, in the Z direction, fine adjustment can be made to help the visual monitoring device 1 focus.

[0044] The visual monitoring device 1 is located directly above the loading platform, including but not limited to: an industrial camera (eg, 20 million pixels) and a 10x objective lens, for collecting images and real-time monitoring of the glass slide 6 and cutting needle 7 on the loading platform.

[0045] The cutting needle control device 2 includes: a power supply, a drive motor, a three-axis motion slider, etc., which controls the cutting needle to achieve high-precision movement in the XYZ three-axis direction according to the coordinate information, and is used to cut tumor tissue. In some embodiments, a pressure sensor is installed in the Z-axis direction of the cutting needle to prevent the cutting needle from crushing the glass slide. The cutting needle 7 can be, but is not limited to, a tungsten steel needle with a diameter of 0.3 mm.

[0046] In some other embodiments, the cutting needle 7 is mounted on an adapter fixture, and metal magnetic points that attract each other are provided on the adapter fixture and the microdissection body. Before microdissection, the adapter fixture with the cutting needle 7 is brought close to the microdissection body, and the two are magnetically aligned due to the magnetic force. The metal magnetic point of the adapter fixture contacts the metal magnetic point of the microdissection body to form a circuit, and the microdissection device can operate.

[0047] The algorithm processing device 3 may include, but is not limited to, a workstation with an Intel Core i9 processor, 32GB memory, and an NVIDIA RTX3090 graphics card, etc. The algorithm processing device 3 has multiple algorithms built in.

[0048] Each of the above steps is described in detail below.

[0049] Step S1 uses a digital microscope to scan the slide to obtain a whole-slice image, namely, a WSI image.

[0050] Step S2 uses the pre-trained tumor recognition model to identify all tumor areas and their locations on the WSI image.

[0051] like Figure 3 As shown in Figure 2, the tumor recognition model will identify all tumor cells on the WSI image and record the area where these tumor cells are located as the tumor area. ) Since WSI-based artificial intelligence tumor recognition is a mature technology, it will not be described in detail here.

[0052] Step S3: Place and fix a glass slide on the sample platform, drip an appropriate amount of paraffin oil on the sample, and fine-tune the position of the sample platform. Through the XY direction, the sample is placed in the visual acquisition area of ​​the visual monitoring device. Through the fine adjustment of the Z axis, the visual monitoring device can focus and collect a clear initial image I0. The initial image I0 shows the slice and the cutting needle, as shown in FIG. Figure 4 shown.

[0053] In step S4, the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image is determined by an image registration algorithm to obtain the tumor region and its position on the initial image I0.

[0054] Specifically, step S4 includes: Step S4.1: Using the initial image I0 as the reference image and the WSI image as the floating image, the initial image I0 and the WSI image are registered by an image registration algorithm (e.g., a registration algorithm based on the sift feature operator), as shown in Figure 5 As shown; Step S4.2: After image registration, determine the position of the initial image I0 on the WSI image, and obtain the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image; Step S4.3: According to the relative relationship between the two coordinate systems, based on all tumor regions and their positions on the WSI image, the tumor region and its position on the initial image I0 are obtained, which is recorded as T0.

[0055] The above-mentioned image registration algorithm is a mature technology, so it will not be described in detail here.

[0056] Step S5 uses a needle recognition algorithm to identify the position of the cutting needle tip on the initial image I0. The needle recognition algorithm may use a template matching method.

[0057] Specifically, step S5 includes: Step S5.1: Using the needle tip image of the cutting needle as a template image, sliding pixel by pixel on the initial image I0, and using a similarity measurement method to calculate and record the matching degree between the template image and the sliding area of ​​the initial image I0 at each position; Step S5.2: Find the position with the highest matching degree, which is the position P0 of the cutting needle in the initial image I0, such as Figure 6 shown.

[0058] In step S6, the needle tip position of the cutting needle on the initial image I0 is used as the initial position, and the position of the tumor area to be cut on the initial image I0 is used as the target position, and the movement path of the cutting needle is obtained by calculation through a path planning algorithm.

[0059] In step S7, the current needle position is taken as the zero position, the cutting needle is lifted, and moves to the target position according to the obtained cutting needle movement path, and then falls down to cut on the corresponding glass slide, with a cutting depth of 3-5 microns.

[0060] In the tumor area T0, the path of the cutting needle is scraped line by line, such as back and forth scraping and unidirectional scraping. As shown in Figure 7 (a), the efficiency of back and forth scraping will be higher, but it may be affected by the gap of the control screw; as shown in Figure 7 (b), the efficiency of unidirectional scraping will be low, but it will effectively avoid the error caused by the gap of the screw.

[0061] During the cutting process, a visual monitoring device is used to obtain real-time images. t , and the real-time image I t Compared with the initial image I0, the deformation of the normal tissue area is obtained. When the deformation exceeds the threshold, the cutting is stopped; otherwise, the cutting is continued until the tumor area is completely cut.

[0062] Specifically, step S7 includes: Step S7.1: During the cutting process, use a visual monitoring device to obtain real-time images I t , the real-time normal tissue area N is obtained according to the following formula t : N t = I t - T0; Where: T0 is the tumor area on the initial image I0; Step S7.2: Obtain the initial normal tissue area N0 according to the following formula: N0= I0- T0; Step S7.3: Calculate the real-time normal tissue area N t The average value of the sum of squares of the differences between the initial normal tissue area N0 and the corresponding pixel values ​​is used to obtain the deformation of the normal tissue area, and the deformation is compared with the threshold. When the deformation exceeds the threshold, the cutting is stopped. Step S7.4: Repeat steps S7.1 to S7.3 until the current tumor area is cut.

[0063] The present invention calculates the mean square error (MSE) between the normal tissue area of ​​the real-time image and the normal tissue area of ​​the initial image to understand the deformation of the image. If the deformation is normal, the cutting will continue; if the difference is large, the cutting will be interrupted to prevent further damage to the tissue.

[0064] In step S8, after the cutting of the current tumor area is completed, the slide is moved to the next area by moving the loading platform, and steps S3 to S7 are repeated until all tumor areas on the slide are cut to achieve tumor cell sampling.

[0065] Collect the excised tumor cell tissue for molecular testing.

[0066] In other embodiments, Figure 8 As shown, multiple cutting needles can be used to cooperate with each other for cutting.

[0067] Specifically, step S6 includes: Step S6.1: extracting features from the part of the tumor region to be cut to obtain tumor features, including morphological features (such as area, perimeter, compactness, irregularity of tumor boundary, etc.), texture features (such as microscopic texture features, etc.) and heterogeneity features (such as color difference in multiple regions, uneven nuclear distribution, etc.); Step S6.2: using the trained tumor complexity judgment model to judge the complexity of the tumor area to be cut, and outputting the complexity level T and the number of cutting needles n; For example: low complexity, using 1 cutting needle; Medium complexity, using 2 cutting needles; High complexity, using 3 cutting needles; Step S6.3: selecting n cutting needles closest to the tumor area to be cut as waiting cutting needles, and assigning a corresponding cutting area to each waiting cutting needle; Step S6.4: Taking the needle tip position of each cutting needle to be activated on the initial image I0 as its initial position, and taking the position of the cutting area of ​​the tumor area to be cut on the initial image I0 as its target position, the cutting needle motion path of each cutting needle to be activated is obtained through calculation by the path planning algorithm.

[0068] Of course, in some other embodiments, multiple cutting needles can cut multiple tumor areas at the same time, using a one-to-one allocation method to improve cutting efficiency. Multiple cutting needles give priority to cutting tumor areas close to themselves to prevent mutual interference during the cutting process.

[0069] In other embodiments, Fig. 9 As shown, in order to improve the degree of automation, the present invention can cut multiple glass slides at one time and can use multiple visual monitoring devices.

[0070] The same as the above embodiment, the difference lies in the above step S3.

[0071] Step S3 is to place and fix a plurality of glass slides on a loading platform, use a plurality of visual monitoring devices to acquire images, and splice the acquired images to obtain an initial image I0, on which slices and cutting needles are presented.

[0072] The present invention discloses a microscopic automatic cutting method for tumor cell sampling, which has the following beneficial effects: First, the equipment is low-cost and does not require expensive laser light sources and high-precision microscopes. It only requires ordinary cameras and cutting needles, which greatly reduces the equipment investment cost and is conducive to its promotion and application in more laboratories and research institutions.

[0073] Second, it is easy to operate and does not require long-term professional training. Just place the glass slide on the loading platform to automatically identify tumor cells and plan the cutting path. The operator does not need to be proficient in complex laser parameter settings and sample processing procedures.

[0074] Third, the cutting throughput is high, and the entire slice can be cut automatically, which greatly improves the cutting efficiency and meets the needs of large-scale, high-throughput sample collection.

[0075] Fourth, laser thermal damage is avoided. No laser irradiation is required during the cutting process, and no thermal damage is caused to the sample, thus ensuring the activity of intracellular biomacromolecules and providing reliable materials for subsequent molecular biology detection.

[0076] Fifth, the cutting accuracy is high. With the help of artificial intelligence algorithms, tumor cells on the slices can be accurately identified, and precise cutting can be achieved through path planning algorithms, minimizing contamination from surrounding irrelevant cells and ensuring sample purity. This allows the extracted samples to be tested for macromolecules such as nucleic acids and proteins, and the results are closer to the real state in the body.

[0077] In some other embodiments, the present invention discloses a microscopic automatic cutting device for tumor cell sampling, comprising: A WSI image acquisition module, used to scan the slide using a digital microscope to acquire a WSI image; The tumor region recognition module is used to identify all tumor regions and their locations on the WSI image using a pre-trained tumor recognition model; An initial image acquisition module is used to place and fix one or more glass slides on the loading platform, and use one or more visual monitoring devices to obtain an initial image I0, on which a slice and a cutting needle are presented; An image registration module, used to determine the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image through an image registration algorithm, and obtain the tumor area and its position on the initial image I0; A cutting needle recognition module is used to identify the position of the cutting needle tip on the initial image I0 through a needle tip recognition algorithm; A path planning module is used to obtain a movement path of the cutting needle by using the needle tip position of the cutting needle on the initial image I0 as the initial position and the position of the tumor area to be cut on the initial image I0 as the target position through calculation by a path planning algorithm; A cutting module, used for the cutting needle to move to a target position according to the obtained cutting needle movement path and then fall down to perform cutting on the corresponding glass slide; Cutting monitoring module, used to obtain real-time images using visual monitoring devices during the cutting process t , and the real-time image I t Compared with the initial image I0, the deformation of the normal tissue area is obtained. When the deformation exceeds the threshold, the cutting is stopped; otherwise, the cutting is continued until the current tumor area is cut completely; The repeated execution module is used to move the glass slide after the current tumor area is cut, and then repeat the methods in the initial image acquisition module, image registration module, cutting needle recognition module, path planning module, cutting module, and cutting monitoring module until all tumor areas on the glass slide are cut to achieve tumor cell sampling.

[0078] Furthermore, the image registration module includes: An image registration unit, used for registering the initial image I0 with the WSI image by using an image registration algorithm, taking the initial image I0 as a reference image and the WSI image as a floating image; A coordinate system corresponding unit, used to determine the position of the initial image I0 on the WSI image after image registration, and obtain the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image; The initial image tumor region determining unit is used to obtain the tumor region and its position on the initial image I0 based on the relative relationship between the two coordinate systems and all the tumor regions and their positions on the WSI image.

[0079] As a preferred solution, the cutting needle identification module includes: A template matching unit is used to use the needle head image of the cutting needle as a template image, slide pixel by pixel on the initial image I0, and use a similarity measurement method to calculate and record the matching degree between the template image and the sliding area of ​​the initial image I0 at each position; The cutting needle position determination unit is used to find the position with the highest matching degree, that is, the position of the cutting needle in the initial image I0.

[0080] Further, the cutting monitoring module includes: Real-time normal tissue acquisition unit, used to obtain real-time images using a visual monitoring device during the cutting process t , the real-time normal tissue area N is obtained according to the following formula t : N t = I t - T0; Where: T0 is the tumor area on the initial image I0; The initial normal tissue acquisition unit is used to obtain the initial normal tissue area N0 according to the following formula: N0= I0- T0; Deformation calculation unit, used to calculate the real-time normal tissue area N t The average value of the sum of squares of the differences between the initial normal tissue area N0 and the corresponding pixel values ​​is used to obtain the deformation of the normal tissue area, and the deformation is compared with the threshold. When the deformation exceeds the threshold, the cutting is stopped. The repetitive execution unit is used to repeat the methods in the real-time normal tissue acquisition unit, the initial normal tissue acquisition unit, and the deformation calculation unit until the current tumor area is cut.

[0081] Furthermore, the path planning module includes: A feature extraction unit, used for extracting features from a portion of the tumor area to be cut to obtain tumor features; The complexity recognition unit is used to judge the complexity of the tumor area to be cut using the trained tumor complexity judgment model, and output the complexity level T and the number of cutting needles n; A cutting needle selection unit is used to select n cutting needles closest to the tumor area to be cut as waiting cutting needles, and assign a corresponding cutting area to each waiting cutting needle; The path planning unit is used to obtain the cutting needle motion path of each cutting needle to be activated by using the needle head position of each cutting needle to be activated on the initial image I0 as its initial position and the position of the cutting area of ​​the tumor area to be cut on the initial image I0 as its target position, and calculating through the path planning algorithm.

[0082] Furthermore, it should be noted that: the microscopic automatic cutting device for tumor cell sampling provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when performing tumor cell cutting. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the microscopic automatic cutting device for tumor cell sampling is divided into different functional modules to complete all or part of the functions described above.

[0083] In addition, the micro-automatic cutting device for tumor cell sampling provided in the above embodiment and the micro-automatic cutting method for tumor cell sampling belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0084] The present invention discloses a microscopic automatic cutting method and device for tumor cell sampling, which has the following beneficial effects: First, the present invention can accurately identify the location of the tumor area and achieve high-precision positioning and fully automatic cutting.

[0085] Second, the present invention associates the WSI image with the image collected by the visual monitoring device through an image registration algorithm, and maps the tumor area on the WSI image to the initial image, thereby ensuring high precision and full automation of the cutting process.

[0086] Third, the present invention uses a visual monitoring device to monitor the cutting process, based on the deformation of the normal tissue area, to ensure the stability of the cutting process.

[0087] Fourth, the equipment used in the present invention is low-cost, easy to operate, has a high cutting flux, avoids laser thermal damage, and has precise cutting.

[0088] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for illustrating the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which shall fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A microscopic automatic cutting method for tumor cell sampling, characterized in that: include: Step S1: Scan the slide using a digital microscope to obtain a WSI image; Step S2: using the pre-trained tumor recognition model to identify all tumor areas and their locations on the WSI image; Step S3: placing and fixing one or more glass slides on a loading platform, and using one or more visual monitoring devices to obtain an initial image I0, where a slice and a cutting needle are shown on the initial image I0; Step S4: determining the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image through an image registration algorithm, and obtaining the tumor area and its position on the initial image I0; Step S5: using a needle recognition algorithm, identifying the position of the cutting needle tip on the initial image I0; Step S6: Taking the needle tip position of the cutting needle on the initial image I0 as the initial position and the position of the tumor area to be cut on the initial image I0 as the target position, the movement path of the cutting needle is obtained by calculation through a path planning algorithm; Step S7: the cutting needle moves to the target position according to the obtained cutting needle movement path and then falls down to perform cutting on the corresponding glass slide; During the cutting process, a visual monitoring device is used to obtain real-time images. t , and the real-time image I t Compared with the initial image I0, the deformation of the normal tissue area is obtained. When the deformation exceeds the threshold, the cutting is stopped; otherwise, the cutting is continued until the current tumor area is cut completely; Step S8: After the cutting of the current tumor area is completed, the slide is moved and steps S3 to S7 are repeated until all tumor areas on the slide are cut to achieve tumor cell sampling.

2. The micro-automatic dissection method according to claim 1, characterized in that: The step S4 comprises: Step S4.1: using the initial image I0 as a reference image and the WSI image as a floating image, the initial image I0 and the WSI image are registered by an image registration algorithm; Step S4.2: After image registration, determine the position of the initial image I0 on the WSI image, and obtain the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image; Step S4.3: According to the relative relationship between the two coordinate systems, based on all tumor regions and their positions on the WSI image, the tumor regions and their positions on the initial image I0 are obtained.

3. The micro-automatic dissection method according to claim 1, characterized in that: The step S5 comprises: Step S5.1: Using the needle tip image of the cutting needle as a template image, sliding pixel by pixel on the initial image I0, and using a similarity measurement method to calculate and record the matching degree between the template image and the sliding area of ​​the initial image I0 at each position; Step S5.2: Find the position with the highest matching degree, which is the position of the cutting needle in the initial image I0.

4. The micro-automatic dissection method according to claim 1, characterized in that: The step S7 comprises: Step S7.1: During the cutting process, use a visual monitoring device to obtain real-time images I t , the real-time normal tissue area N is obtained according to the following formula t : N t = I t -T0; Where: T0 is the tumor area on the initial image I0; Step S7.2: Obtain the initial normal tissue area N0 according to the following formula: N0= I0- T0; Step S7.3: Calculate the real-time normal tissue area N t The average value of the sum of squares of the differences between the initial normal tissue area N0 and the corresponding pixel values ​​is used to obtain the deformation of the normal tissue area, and the deformation is compared with the threshold. When the deformation exceeds the threshold, the cutting is stopped. Step S7.4: Repeat steps S7.1 to S7.3 until the current tumor area is cut.

5. The micro-automatic dissection method according to claim 1, characterized in that: The step S6 comprises: Step S6.1: extracting features from the part of the tumor area to be cut to obtain tumor features; Step S6.2: using the trained tumor complexity judgment model to judge the complexity of the tumor area to be cut, and outputting the complexity level T and the number of cutting needles n; Step S6.3: selecting n cutting needles closest to the tumor area to be cut as waiting cutting needles, and assigning a corresponding cutting area to each waiting cutting needle; Step S6.4: Taking the needle tip position of each cutting needle to be activated on the initial image I0 as its initial position, and taking the position of the cutting area of ​​the tumor area to be cut on the initial image I0 as its target position, the cutting needle motion path of each cutting needle to be activated is obtained through calculation by the path planning algorithm.

6. A microscopic automatic cutting device for tumor cell sampling, characterized in that: include: A WSI image acquisition module, used to scan the slide using a digital microscope to acquire a WSI image; The tumor region recognition module is used to identify all tumor regions and their locations on the WSI image using a pre-trained tumor recognition model; An initial image acquisition module is used to place and fix one or more glass slides on the loading platform, and use one or more visual monitoring devices to obtain an initial image I0, on which a slice and a cutting needle are presented; An image registration module, used to determine the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image through an image registration algorithm, and obtain the tumor area and its position on the initial image I0; A cutting needle recognition module is used to identify the position of the cutting needle tip on the initial image I0 through a needle tip recognition algorithm; A path planning module is used to obtain a movement path of the cutting needle by using the needle tip position of the cutting needle on the initial image I0 as the initial position and the position of the tumor area to be cut on the initial image I0 as the target position through calculation by a path planning algorithm; A cutting module, used for the cutting needle to move to a target position according to the obtained cutting needle movement path and then fall down to perform cutting on the corresponding glass slide; Cutting monitoring module, used to obtain real-time images using visual monitoring devices during the cutting process t , and the real-time image I t Compared with the initial image I0, the deformation of the normal tissue area is obtained. When the deformation exceeds the threshold, the cutting is stopped; otherwise, the cutting is continued until the current tumor area is cut completely; The repeated execution module is used to move the glass slide after the current tumor area is cut, and then repeat the methods in the initial image acquisition module, image registration module, cutting needle recognition module, path planning module, cutting module, and cutting monitoring module until all tumor areas on the glass slide are cut to achieve tumor cell sampling.

7. The microautomatic dissection device according to claim 6, characterized in that: The image registration module comprises: An image registration unit, used for registering the initial image I0 with the WSI image by using an image registration algorithm, taking the initial image I0 as a reference image and the WSI image as a floating image; A coordinate system corresponding unit, used to determine the position of the initial image I0 on the WSI image after image registration, and obtain the relative relationship between the coordinate system of the initial image I0 and the coordinate system of the WSI image; The initial image tumor region determining unit is used to obtain the tumor region and its position on the initial image I0 based on the relative relationship between the two coordinate systems and all the tumor regions and their positions on the WSI image.

8. The microautomatic dissection device according to claim 6, characterized in that: The cutting needle identification module comprises: The template matching unit is used to use the needle head image of the cutting needle as a template image, slide pixel by pixel on the initial image I0, and calculate and record the matching degree between the template image and the sliding area of ​​the initial image I0 at each position by using a similarity measurement method; The cutting needle position determination unit is used to find the position with the highest matching degree, that is, the position of the cutting needle in the initial image I0.

9. The microautomatic dissection device according to claim 6, characterized in that: The cutting monitoring module comprises: Real-time normal tissue acquisition unit, used to obtain real-time images using a visual monitoring device during the cutting process t , the real-time normal tissue area N is obtained according to the following formula t : N t = I t -T0; Where: T0 is the tumor area on the initial image I0; The initial normal tissue acquisition unit is used to obtain the initial normal tissue area N0 according to the following formula: N0= I0- T0; Deformation calculation unit, used to calculate the real-time normal tissue area N t The average value of the sum of squares of the differences between the initial normal tissue area N0 and the corresponding pixel values ​​is used to obtain the deformation of the normal tissue area, and the deformation is compared with the threshold. When the deformation exceeds the threshold, the cutting is stopped. The repetitive execution unit is used to repeat the methods in the real-time normal tissue acquisition unit, the initial normal tissue acquisition unit, and the deformation calculation unit until the current tumor area is cut.

10. The microautomatic dissection device according to claim 6, characterized in that: The path planning module includes: A feature extraction unit, used for extracting features from a portion of the tumor area to be cut to obtain tumor features; The complexity recognition unit is used to judge the complexity of the tumor area to be cut using the trained tumor complexity judgment model, and output the complexity level T and the number of cutting needles n; A cutting needle selection unit is used to select n cutting needles closest to the tumor area to be cut as waiting cutting needles, and assign a corresponding cutting area to each waiting cutting needle; The path planning unit is used to obtain the cutting needle motion path of each cutting needle to be activated by using the needle head position of each cutting needle to be activated on the initial image I0 as its initial position and the position of the cutting area of ​​the tumor area to be cut on the initial image I0 as its target position, and calculating through the path planning algorithm.

Images