Needle insertion guide method, apparatus, device, and storage medium

Abstract

This invention relates to the field of medical technology and discloses a needle insertion guidance method, device, equipment, and storage medium. The needle insertion guidance method includes: importing medical image data; performing three-dimensional reconstruction based on the medical image data to obtain a three-dimensional model corresponding to the medical image data; planning the needle insertion path information of the electrode needle based on a virtual target location; generating a virtual reference plane where the limiting end of the electrode needle is located based on the needle insertion path information; obtaining the pose adjustment information of the robotic arm based on the needle insertion path information and the virtual reference plane; driving the robotic arm to adjust to the needle insertion pose based on the pose adjustment information, wherein the pinhole array plate at the end of the robotic arm corresponds to the virtual reference plane during the needle insertion pose; after the electrode needle is installed on the pinhole array plate and the limiting end abuts against the pinhole array plate, an ablation signal is transmitted to the electrode needle, causing the electrode needle to ablate the real target location. This invention reconstructs the needle insertion plane, enabling the robotic arm to accurately move to the needle insertion pose, improving the accuracy of needle insertion and the success rate of ablation surgery.

Description

Technical Field

[0001] This invention relates to the field of medical technology, and in particular to a needle insertion guidance method, device, equipment, and storage medium. Background Technology

[0002] During ablation surgery, multiple electrode needles are usually used in combination to ensure that the target organ and the target lesion area can be fully ablated.

[0003] However, in current ablation procedures, the low precision of manual needle insertion makes it difficult to control the relative positions of multiple needles. This necessitates multiple insertions to adjust the angle and position, forcing patients to endure repeated needle insertions, causing unnecessary physical damage, and increasing surgical risks. Furthermore, current ablation procedures lack 3D algorithms for assistance, making it difficult for surgeons to determine the needle insertion point and plan the insertion path. Summary of the Invention

[0004] This invention provides a needle insertion guidance method, device, equipment, and storage medium, which can accurately determine the needle insertion position and accurately plan the needle insertion path, improve the accuracy of needle insertion, increase the success rate of ablation surgery, and reduce the risks in ablation surgery.

[0005] In a first aspect, embodiments of the present invention provide a needle insertion guidance method, comprising: importing medical image data; performing three-dimensional reconstruction based on the medical image data to obtain a three-dimensional model corresponding to the medical image data, the three-dimensional model including a virtual target area, the virtual target area corresponding to a real target area to be ablated; planning the needle insertion path information of an electrode needle based on the virtual target area, the electrode needle having an ablation end and a limiting end; generating a virtual reference surface where the limiting end of the electrode needle is located based on the needle insertion path information; obtaining pose adjustment information of a robotic arm based on the needle insertion path information and the virtual reference surface; driving the robotic arm to adjust to the needle insertion pose based on the pose adjustment information, wherein the pinhole array plate at the end of the robotic arm corresponds to the virtual reference surface during the needle insertion pose; after the electrode needle is installed on the pinhole array plate and the limiting end abuts against the pinhole array plate, transmitting an ablation signal to the electrode needle, thereby causing the electrode needle to ablate the real target area.

[0006] According to the foregoing embodiments of the first aspect of the present invention, the medical imaging data includes multiple two-dimensional cross-sectional images of a real target site. Three-dimensional reconstruction based on the medical imaging data to obtain a three-dimensional model corresponding to the medical imaging data includes: determining a designated target point in one of the two-dimensional cross-sectional images that falls within a target region, where the target region is the corresponding region of the real target site in the two-dimensional cross-sectional image; obtaining multiple two-dimensional contour feature information based on the designated target point and the multiple two-dimensional cross-sectional images, where each two-dimensional contour feature information characterizes the contour of the target region in the corresponding two-dimensional cross-sectional image; and obtaining a three-dimensional model based on the multiple two-dimensional contour feature information.

[0007] According to the foregoing embodiments of the first aspect of the present invention, obtaining multiple two-dimensional contour feature information based on a specified target point and multiple two-dimensional cross-sectional images includes: mapping a specified target point to a mapped target point in the remaining two-dimensional cross-sectional images; and in each two-dimensional cross-sectional image, obtaining two-dimensional contour feature information based on the specified target point or the mapped target point using a maximum connected component algorithm.

[0008] According to the aforementioned embodiments of the first aspect of the present invention, the two-dimensional contour feature information is a set of points arranged along the contour of the target region, and obtaining a three-dimensional model based on multiple two-dimensional contour feature information includes: performing point cloud three-dimensional reconstruction on the multiple two-dimensional contour feature information to obtain a three-dimensional model.

[0009] According to the aforementioned embodiment of the first aspect of the present invention, the needle insertion path information includes the needle insertion path information of at least one electrode needle, and in the step of generating a virtual reference plane where the limiting end of the electrode needle is located based on the needle insertion path information, the virtual reference plane is perpendicular to each electrode needle.

[0010] According to the aforementioned embodiments of the first aspect of the present invention, the planning of the electrode needle insertion path information based on the three-dimensional model of the target lesion includes: obtaining the electrode needle model in the coordinate system of the three-dimensional model; obtaining three-dimensional line segments, wherein the three-dimensional line segments are the motion paths connecting the electrode needle model and the virtual target site; performing a collision test on the electrode needle model and the three-dimensional model according to the three-dimensional line segments to determine whether a collision event exists; if a collision event exists, adjusting the colliding three-dimensional line segments according to the obtained adjustment instructions; and using the starting point, ending point, and trajectory of the three-dimensional line segments as the insertion starting point, insertion ending point, and insertion route of the electrode needle to obtain the insertion path information.

[0011] According to the aforementioned embodiments of the first aspect of the present invention, obtaining the pose adjustment information of the robotic arm based on the needle insertion path information and the virtual reference plane includes: calibrating the coordinate systems of the optical positioning device and the robotic arm to obtain a first transformation matrix of the coordinate systems of the optical positioning device and the robotic arm; calibrating the coordinate systems of the three-dimensional model and the robotic arm to obtain a second transformation matrix of the coordinate systems of the three-dimensional model and the robotic arm; obtaining the pre-adjustment pose of the robotic arm through the optical positioning device; obtaining the needle insertion pose of the robotic arm through the needle insertion path information and the virtual reference plane; and obtaining the pose adjustment information based on the first transformation matrix, the second transformation matrix, the pre-adjustment pose, and the needle insertion pose.

[0012] Secondly, embodiments of the present invention provide a needle insertion guidance device, comprising: a data import module for importing medical image data; a three-dimensional reconstruction module for performing three-dimensional reconstruction based on the medical image data to obtain a three-dimensional model corresponding to the medical image data, the three-dimensional model including a virtual target area, the virtual target area corresponding to the real target area to be ablated; a needle insertion planning module for planning the needle insertion path information of an electrode needle based on the three-dimensional model of the target lesion, the electrode needle having an ablation end and a limiting end; a robotic arm pose planning module for generating a virtual reference surface where the limiting end of the electrode needle is located based on the needle insertion path information, and for obtaining pose adjustment information of the robotic arm based on the needle insertion path information and the virtual reference surface; a robotic arm driving module for driving the robotic arm to adjust to the needle insertion pose based on the pose adjustment information, wherein the pinhole array plate at the end of the robotic arm corresponds to the virtual reference surface during the needle insertion pose; and an electrode needle control module for transmitting an ablation signal to the electrode needle after the electrode needle is installed on the pinhole array plate and the limiting end abuts against the pinhole array plate, so that the electrode needle ablates the real target area.

[0013] Thirdly, embodiments of the present invention provide a needle insertion guiding device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the needle insertion guiding device to execute the needle insertion guiding method of any of the foregoing embodiments of the first aspect of the present invention.

[0014] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the needle insertion guidance method of any of the foregoing embodiments of the first aspect of the present invention.

[0015] According to the technical solution of the needle insertion guidance method provided by the present invention, a three-dimensional model of a corresponding virtual target area can be generated based on medical imaging data. The needle insertion path information of the electrode needle is planned based on the three-dimensional model of the virtual target area. A virtual reference surface where the limiting end of the electrode needle is located is generated based on the needle insertion path information and the virtual reference surface. The pose adjustment information of the robotic arm is obtained based on the needle insertion path information and the virtual reference surface. The robotic arm is adjusted to the needle insertion pose based on the pose adjustment information. In the needle insertion pose, the pinhole array plate at the end of the robotic arm corresponds to the virtual reference surface. After the electrode needle is installed on the pinhole array plate and the limiting end abuts against the pinhole array plate, the electrode needle is controlled to ablate the real target area. In this embodiment of the invention, the three-dimensional algorithm can accurately determine the needle insertion position and accurately plan the needle insertion path, improving the accuracy of needle insertion, increasing the success rate of ablation surgery, and reducing the risks during ablation surgery. Attached Figure Description

[0016] Figure 1 This is a flowchart of one embodiment of the needle insertion guidance method in this invention;

[0017] Figure 2 This is a flowchart of the steps in one embodiment of the needle insertion guidance method of the present invention to perform three-dimensional reconstruction based on medical image data to obtain a three-dimensional model corresponding to the medical image data;

[0018] Figure 3 This is a flowchart of the steps in one embodiment of the needle insertion guidance method of the present invention, which involves planning the needle insertion path information of the electrode needle based on a three-dimensional model of the target lesion.

[0019] Figure 4 This is a flowchart of the steps in one embodiment of the needle insertion guidance method of the present invention to obtain the pose adjustment information of the robotic arm based on the needle insertion path information and the virtual reference plane;

[0020] Figure 5 This is a schematic diagram of an embodiment of the needle insertion guide device of the present invention;

[0021] Figure 6 This is a schematic diagram of the structure of a needle insertion guide device provided in an embodiment of the present invention. Detailed Implementation

[0022] This invention provides a needle insertion guidance method, device, equipment, and storage medium. It can generate a three-dimensional model of a corresponding virtual target area based on medical imaging data. Based on the three-dimensional model of the virtual target area, it plans the insertion path information of the electrode needle. Based on the insertion path information, it generates a virtual reference surface where the limiting end of the electrode needle is located. Based on the insertion path information and the virtual reference surface, it obtains the pose adjustment information of a robotic arm. Based on the pose adjustment information, it adjusts the robotic arm to the needle insertion pose. In the needle insertion pose, the pinhole array plate at the end of the robotic arm corresponds to the virtual reference surface. After the electrode needle is installed on the pinhole array plate and the limiting end abuts against the pinhole array plate, an ablation signal is transmitted to the electrode needle, causing the electrode needle to ablate the real target area. In this invention, through a three-dimensional algorithm, the needle insertion position can be accurately determined and the insertion path can be accurately planned, improving the accuracy of needle insertion, increasing the success rate of ablation surgery, and reducing the risks during ablation surgery.

[0023] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] For ease of understanding, the specific process of the embodiments of the present invention will be described below.

[0025] Figure 1 This is a flowchart of an embodiment of the needle insertion guidance method of the present invention, which includes steps S110 to S170.

[0026] In step S110, medical imaging data is imported.

[0027] In some embodiments, the patient's medical imaging data is primarily DICOM (Digital Imaging and Communications in Medicine) images. DICOM images refer to medical images stored according to the DICOM standard. The DICOM standard is a standard for data storage and communication transmission between medical imaging devices, defining a medical image format that meets clinical needs and can be used for data exchange. After scanning a patient using medical imaging equipment, DICOM images of different thicknesses are typically obtained. Each thickness of DICOM image is categorized by section (cross-section, sagittal, coronal), and each section is categorized by stage (venous phase, arterial phase, equilibrium phase, etc.). DICOM images of the same patient, of the same thickness, from the same section, and at the same stage have the same serial number. The serial number of DICOM images of the same patient indicates the scanning order, and the instance number of DICOM images with the same series serial number indicates the image generation order.

[0028] In some embodiments, DICOM images can be directly copied from a CT scanner / MRI scanner / hospital imaging workstation using a USB flash drive. Then, the corresponding directory can be selected from the transfer file to copy the DICOM image to the corresponding software directory. By reading the files in the software directory, the corresponding DICOM image can be obtained.

[0029] In step S120, three-dimensional reconstruction is performed based on the medical imaging data to obtain a three-dimensional model corresponding to the medical imaging data. The three-dimensional model includes a virtual target area, which corresponds to the real target area to be ablated. In some embodiments, the real target area is, for example, an organ or a lesion.

[0030] In some embodiments, before performing 3D reconstruction on medical image data, i.e., DICOM images, all DICOM images can be filtered to select the DICOM image of interest. The DICOM image of interest can be identified by keywords in the patient's medical record, such as the diseased organ and lesion, like the liver or hilar tumor, or it can be the target organ and lesion directly input by the user. This invention is not limited to these specific examples. In some embodiments, the images can be preprocessed using techniques such as noise reduction, compression, segmentation, registration, and fusion.

[0031] Image 3D reconstruction is divided into surface rendering and volume rendering. In this embodiment, the description mainly focuses on surface rendering. The present invention does not limit the method of 3D reconstruction. In practical applications, there are many surface rendering algorithms, and the present invention does not limit them. In this embodiment, the maximum connected component algorithm is mainly used for rendering. The maximum connected component algorithm is used to calculate the 3D model corresponding to the 3D reconstructed DICOM image, including the virtual target area.

[0032] In some embodiments, the coordinate system of the virtual target area corresponds to that of the real target area to be ablated, and the virtual target area is the real target area to be ablated in the DICOM image.

[0033] Figure 2 This is a flowchart illustrating the steps of performing three-dimensional reconstruction based on medical imaging data to obtain a three-dimensional model corresponding to the medical imaging data in one embodiment of the needle insertion guidance method of the present invention. In some embodiments, the medical imaging data includes multiple two-dimensional cross-sectional images of the actual target area. In this case, step S120 of performing three-dimensional reconstruction based on the medical imaging data to obtain a three-dimensional model corresponding to the medical imaging data may include steps S121 to S123.

[0034] In step S121, a designated target point that falls into the target area is determined in one of the two-dimensional cross-sectional images. The target area is the corresponding area of ​​the actual target part in the two-dimensional cross-sectional image.

[0035] In step S122, multiple two-dimensional contour feature information is obtained based on the specified target point and multiple two-dimensional cross-sectional images. Each two-dimensional contour feature information represents the contour of the target region in the corresponding two-dimensional cross-sectional image.

[0036] In step S123, a three-dimensional model is obtained based on multiple two-dimensional contour feature information.

[0037] Step S122, which obtains multiple two-dimensional contour feature information based on a specified target point and multiple two-dimensional cross-sectional images, may include steps S1221 to S1222.

[0038] In step S1221, the mapped target point is obtained by mapping the specified target point in the remaining two-dimensional cross-sectional image.

[0039] In step S1222, in each two-dimensional cross-sectional image, the two-dimensional contour feature information is obtained by using the maximum connected component algorithm based on the specified target point or the mapped target point.

[0040] In some embodiments, the two-dimensional contour feature information is a set of points arranged along the contour of the target region. Step S123, which obtains a three-dimensional model based on multiple two-dimensional contour feature information, may include: step S1231.

[0041] In step S1231, the two-dimensional contour feature information is used to perform point cloud three-dimensional reconstruction to obtain a three-dimensional model.

[0042] In step S130, based on the three-dimensional model of the virtual target area, the insertion path information of the electrode needle is planned, and the electrode needle has an ablation end and a limiting end.

[0043] In some embodiments, the needle insertion path information includes the needle insertion path information of at least one electrode needle, and in the step of generating a virtual reference plane where the limiting end of the electrode needle is located based on the needle insertion path information, the virtual reference plane is perpendicular to each electrode needle.

[0044] Figure 3 This is a flowchart illustrating the steps of planning the insertion path information of the electrode needle based on a three-dimensional model of a virtual target area in one embodiment of the needle insertion guidance method of the present invention. Step S130, which plans the insertion path information of the electrode needle based on the three-dimensional model of the virtual target area, may include steps S131 to S135.

[0045] In step S131, the electrode needle model in the coordinate system of the three-dimensional model is obtained.

[0046] In step S132, a three-dimensional line segment is obtained, which is the motion path connecting the electrode needle model and the virtual target part.

[0047] In step S133, a collision test is performed between the electrode needle model and the three-dimensional model based on the three-dimensional line segment to determine whether a collision event exists.

[0048] In step S134, if a collision event occurs, the three-dimensional line segment involved in the collision is adjusted according to the obtained adjustment instructions.

[0049] In step S135, the starting point, ending point, and trajectory of the three-dimensional line segment are used as the starting point, ending point, and insertion route of the electrode needle to obtain the insertion path information.

[0050] In some embodiments, the needle insertion path information can be obtained by acquiring virtual target site data and using blood vessels, bones, preset organs that prohibit electrode needles from passing through, and the already inserted electrode needle body near the virtual target site as obstacle constraints, taking the real target site corresponding to the virtual target site as the needle insertion endpoint, and taking the epithelial region near the real target site corresponding to the virtual target site as the needle insertion starting point.

[0051] In some embodiments, the needle insertion path information is obtained primarily by placing three-dimensional line segments in a 3D model reconstructed from a 3D image. First, the target organ and lesion (e.g., liver, hilar tumor) are selected as the endpoint of the line segment, and the needle insertion point is selected as the starting point. Then, the direction and position of the line segment are adjusted in the coronal, sagittal, and transverse planes by clicking and dragging the two endpoints of the line segment. During the adjustment process, each time the mouse is lifted, a Boolean operation is performed on the electrode needle model and a preset 3D model representing an obstacle constraint (i.e., an obstacle region). The preset 3D model representing the obstacle constraint may include bones or organs that prevent the needle path from passing through, as well as other needles. Based on the Boolean operation result, it is determined whether a collision has occurred. If a collision occurs, the 3D line segment at the collision location is displayed in red; otherwise, it is displayed in green. Finally, a 3D vector is formed in the image space, and the starting point, ending point, and direction of this 3D vector constitute the needle insertion path information.

[0052] In some embodiments, the ablation range of the electrode needle is calculated at the insertion point of the electrode needle according to preset configuration parameters. In some embodiments, parameters are configured for each electrode needle in advance, and an ellipsoidal ablation range is generated at the needle tip according to the configured parameters. Then, the intersection of this ellipsoidal range with the target area is calculated, and the specific treatment effect, such as the complete ablation range or the unablated range, is evaluated based on the intersection.

[0053] In some embodiments, the unablated volume of the lesion and the overablated volume of the target organ are simulated and calculated based on the volume of the spatial intersection.

[0054] In some embodiments, the spatial intersection of the lesion and the ablation range is taken. This spatial intersection represents the space within the lesion where ablation is performed. After simulation calculations of the electrode needle, the area in the 3D model of the lesion that does not spatially intersect with the ablation range of the electrode needle is the unablated area, and its volume is the unablated volume. Within the ablation range, the area that does not spatially intersect with the 3D model of the lesion is the over-ablated area, and its volume is the over-ablated volume. The ablation strategy is evaluated based on the volume of the spatial intersection, the unablated volume, and the over-ablated volume to obtain the corresponding ablation evaluation results.

[0055] In some embodiments, the ablation strategy can be adjusted based on the ablation assessment results. For example, if the calculated volume of the spatial intersection is smaller than the over-ablation volume during needle insertion planning, it may be that the selection of the needle insertion endpoint in the electrode needle insertion strategy is incorrect. The needle insertion endpoint may be too far from the lesion area to be ablated, or it may be at the edge of the lesion area instead of in the center of the lesion area. This will lead to an unsatisfactory ablation effect. Therefore, the needle insertion strategy can be adjusted based on the relationship between the volume of the spatial intersection, the unablated volume, and the over-ablation volume to further reduce the risk of multiple needle insertions. The steps for adjusting the ablation strategy based on the ablation assessment results may include: determining whether the unablated volume is less than a preset threshold; if the unablated volume is not less than the preset threshold, determining the unablated region corresponding to the unablated volume, and performing a secondary needle insertion plan based on the unablated region to obtain a secondary needle insertion strategy; calculating the ablation range of the secondary needle insertion plan at the insertion endpoint of the secondary needle insertion strategy according to the configuration parameters; calculating the quadratic spatial intersection of the ablation range of the secondary needle insertion plan and the unablated region, and determining whether the unablated volume of the secondary needle insertion plan is less than a preset threshold based on the quadratic spatial intersection; if the unablated volume of the secondary needle insertion plan is not less than the preset threshold, returning to the step of determining the unablated region corresponding to the unablated volume, until the unablated volume is less than the preset threshold.

[0056] In some embodiments, if the volume of the unablated area is too large, it indicates that the ablation effect is not good. This may be due to an inappropriate selection of the needle insertion endpoint or an excessively large volume of the lesion's three-dimensional model. In such cases, multiple simulated needle insertion plans are required. After each needle insertion plan, the unablated volume is calculated and a threshold is set. This threshold can be set according to actual needs. For example, setting it to zero indicates that the lesion area needs to be completely ablated.

[0057] In some embodiments, different ablation schemes may be required depending on the needs of different patients. There may be lesions in different locations, or the lesion area may be large, and the electrode needle may not be able to complete the ablation scheme well in one session, requiring multiple ablations. When calculating the needle insertion strategy, the user can set the remaining needle insertion count and add the remaining needle insertion count to the needle insertion strategy. If different areas need to be ablated, the needle insertion path, needle insertion start point, and needle insertion end point are recalculated each time the robotic arm returns to the initial position and the remaining needle insertion count is updated to be non-zero, until the remaining needle insertion count is zero, indicating that the ablation process has ended, and the needle insertion guidance process ends.

[0058] This invention adds an evaluation of ablation strategies and a scheme for multiple needle insertions based on the evaluation results. By planning the needle insertion once, the number of needle insertions can be reduced. However, in cases where the lesion volume is large, a single needle ablation may not be effective, requiring multiple ablations of different areas of the lesion. By adding multiple ablation schemes during the simulated ablation process and evaluating the ablation process, doctors can adjust the needle insertion plan multiple times based on the evaluation results, resulting in better ablation effects.

[0059] In step S140, a virtual reference plane is generated based on the needle insertion path information to determine the location of the limiting end of the electrode needle.

[0060] In some embodiments, the number of electrode needles inserted is not limited, and those skilled in the art can determine the number as needed. By confirming the starting point and ending point of the electrode needle insertion, a virtual reference plane can be obtained where the limiting end of the electrode needle is located. Multiple electrode needles are of the same size, length, and model, and the virtual reference plane is perpendicular to each electrode needle.

[0061] In step S150, the pose adjustment information of the robotic arm is obtained based on the needle insertion path information and the virtual reference plane.

[0062] Figure 4 This is a flowchart illustrating the steps of obtaining robotic arm pose adjustment information based on needle insertion path information and a virtual reference plane in one embodiment of the needle insertion guidance method of the present invention. Step S150, which obtains robotic arm pose adjustment information based on needle insertion path information and a virtual reference plane, may include steps S151 to S155.

[0063] In step S151, the coordinate systems of the optical positioning device and the robotic arm are calibrated to obtain the first transformation matrix of the coordinate systems of the optical positioning device and the robotic arm.

[0064] In step S152, the coordinate systems of the 3D model and the robotic arm are calibrated to obtain the second transformation matrix of the coordinate systems of the 3D model and the robotic arm.

[0065] In step S153, the pre-adjustment pose of the robotic arm is obtained by an optical positioning device.

[0066] In step S154, the needle insertion pose of the robotic arm is obtained through the needle insertion path information and the virtual reference plane.

[0067] In step S155, pose adjustment information is obtained based on the first transformation matrix, the second transformation matrix, the pre-adjustment pose, and the needle insertion pose.

[0068] In some embodiments, the robotic arm's needle insertion path is calculated based on needle insertion path information. Furthermore, avoidance zones can be specified to mitigate potential risks during robotic arm movement, making the entire surgical procedure safer and more reliable. When the robotic arm's needle insertion path is detected to have passed through an avoidance zone during a needle insertion path change, the path is readjusted until the robotic arm's needle insertion path no longer passes through the avoidance zone. Throughout the entire robotic arm movement, no needles are carried; only the necessary clamps for holding the needles are used.

[0069] In step S160, the robotic arm is driven to adjust to the needle insertion posture according to the posture adjustment information. In the needle insertion posture, the pinhole array plate at the end of the robotic arm corresponds to the virtual reference surface.

[0070] In some embodiments, when the robotic arm adjusts to the needle insertion posture according to the posture adjustment information, the pinhole array plate at the end of the robotic arm is adjusted to correspond to the virtual reference surface according to the coordinate information of the virtual reference surface.

[0071] In step S170, after the electrode needle is installed on the pinhole array plate and the limiting end abuts against the pinhole array plate, the ablation signal is transmitted to the electrode needle so that the electrode needle ablates the actual target area.

[0072] In some embodiments, when the pinhole array board is adjusted to correspond to the virtual reference surface according to the coordinate information of the virtual reference surface, the relevant technicians confirm the insertion position of the electrode needle on the pinhole array board according to the needle insertion path information, and insert the electrode needle into the corresponding pinhole.

[0073] In this embodiment of the invention, a three-dimensional algorithm can accurately determine the needle insertion position and plan the needle insertion path, thereby improving the accuracy of needle insertion, increasing the success rate of ablation surgery, and reducing the risks during ablation surgery.

[0074] The needle insertion guiding method in the embodiments of the present invention has been described above. The needle insertion guiding device in the embodiments of the present invention will be described below. Please refer to [link / reference]. Figure 5 One embodiment of the needle insertion guide device of the present invention includes a data import module 201, a three-dimensional reconstruction module 202, a needle insertion planning module 203, a robotic arm pose planning module 204, a robotic arm drive module 205, and an electrode needle control module 206.

[0075] The data import module 201 is used to import medical image data. The 3D reconstruction module 202 is used to perform 3D reconstruction based on the medical image data to obtain a 3D model corresponding to the medical image data. The 3D model includes a virtual target area, which corresponds to the real target area to be ablated. The needle insertion planning module 203 is used to plan the needle insertion path information of the electrode needle based on the 3D model of the target lesion. The electrode needle has an ablation end and a limiting end. The robotic arm pose planning module 204 is used to generate a virtual reference plane where the limiting end of the electrode needle is located based on the needle insertion path information, and to obtain the pose adjustment information of the robotic arm based on the needle insertion path information and the virtual reference plane. The robotic arm drive module 205 is used to drive the robotic arm to adjust to the needle insertion pose based on the pose adjustment information. In the needle insertion pose, the pinhole array plate at the end of the robotic arm corresponds to the virtual reference plane. The electrode needle control module 206 is used to control the electrode needle to ablate the real target area after the electrode needle is installed on the pinhole array plate and the limiting end abuts against the pinhole array plate.

[0076] According to an embodiment of the needle insertion guidance device provided by the present invention, a three-dimensional model of a corresponding virtual target area can be generated based on medical imaging data. The needle insertion path information of the electrode needle is planned based on the three-dimensional model of the virtual target area. A virtual reference surface where the limiting end of the electrode needle is located is generated based on the needle insertion path information and the virtual reference surface. The pose adjustment information of the robotic arm is obtained based on the needle insertion path information and the virtual reference surface. The robotic arm is adjusted to the needle insertion pose based on the pose adjustment information. In the needle insertion pose, the pinhole array plate at the end of the robotic arm corresponds to the virtual reference surface. After the electrode needle is installed on the pinhole array plate and the limiting end abuts against the pinhole array plate, the electrode needle is controlled to ablate the real target area. In one embodiment of the needle insertion guidance device provided by the present invention, the image acquisition and processing tasks are configured in the same computer, simplifying the software architecture and improving the applicability of the software. Furthermore, through the three-dimensional algorithm, the needle insertion position can be accurately determined and the needle insertion path can be accurately planned, improving the accuracy of needle insertion, increasing the success rate of ablation surgery, and reducing the risks during ablation surgery.

[0077] above Figure 5 The needle insertion guide device in the embodiments of the present invention will be described in detail from the perspective of modular functional entities. The needle insertion guide device in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0078] Figure 6This is a schematic diagram of a needle insertion guiding device provided in an embodiment of the present invention. The needle insertion guiding device 300 can vary significantly due to different configurations or performance characteristics. It may include a processor (central processing unit, CPU) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the needle insertion guiding device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the needle insertion guiding device 300.

[0079] The needle insertion guidance device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 6 The needle guide device structure shown does not constitute a limitation on the needle guide device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0080] The present invention also provides a needle insertion guiding device. The computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs the steps of the needle insertion guiding method described in the above embodiments.

[0081] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the pin insertion method.

[0082] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0083] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A needle insertion guide device, characterized in that, include: The data import module is used to import medical imaging data. The three-dimensional reconstruction module is used to perform three-dimensional reconstruction based on the medical image data to obtain a three-dimensional model corresponding to the medical image data. The three-dimensional model includes a virtual target area, which corresponds to the real target area to be ablated. The needle insertion planning module is used to plan the insertion path information of the electrode needle based on the three-dimensional model of the virtual target area. The electrode needle has an ablation end and a limiting end. The robotic arm pose planning module is used to generate a virtual reference surface where the limiting end of the electrode needle is located based on the needle insertion path information, and to obtain the pose adjustment information of the robotic arm based on the needle insertion path information and the virtual reference surface. The robotic arm drive module is used to drive the robotic arm to adjust to the needle insertion posture according to the posture adjustment information. In the needle insertion posture, the pinhole array plate at the end of the robotic arm corresponds to the virtual reference surface. as well as An electrode needle control module is used to transmit an ablation signal to the electrode needle after the electrode needle is installed on the pinhole array plate and the limiting end abuts against the pinhole array plate, so that the electrode needle ablates the actual target area.

2. The needle guide device according to claim 1, characterized in that, The medical imaging data includes multiple two-dimensional cross-sectional images of the actual target area. The three-dimensional reconstruction module is also used for: In one of the two-dimensional cross-sectional images, a designated target point that falls into the target region is determined, wherein the target region is the corresponding region of the actual target part in the two-dimensional cross-sectional image; Multiple two-dimensional contour feature information is obtained based on the specified target point and the multiple two-dimensional cross-sectional images, and each of the two-dimensional contour feature information represents the contour of the target region in the corresponding two-dimensional cross-sectional image; The three-dimensional model is obtained based on the multiple two-dimensional contour feature information.

3. The needle guide device according to claim 2, characterized in that, The three-dimensional reconstruction module is also used for: The mapped target point is obtained by mapping the specified target point in the remaining two-dimensional cross-sectional image; In each of the two-dimensional cross-sectional images, the two-dimensional contour feature information is obtained by using the maximum connected component algorithm based on the specified target point or the mapped target point.

4. The needle guide device according to claim 3, characterized in that, The two-dimensional contour feature information is a set of points arranged along the contour of the target region. The 3D reconstruction module is also used to: perform point cloud 3D reconstruction on the 2D contour feature information to obtain the 3D model.

5. The needle guide device according to claim 1, characterized in that, The needle insertion path information includes the needle insertion path information of at least one electrode needle. The virtual reference plane is perpendicular to each of the electrode needles.

6. The needle guide device according to claim 1, characterized in that, The needle insertion planning module is also used for: Obtain the electrode needle model in the coordinate system of the three-dimensional model; Obtain a three-dimensional line segment, which is the motion path connecting the electrode needle model and the virtual target part; Based on the three-dimensional line segments, a collision test is performed between the electrode needle model and the three-dimensional model to determine whether a collision event exists. If a collision event occurs, the 3D line segment involved in the collision will be adjusted according to the obtained adjustment instructions. The starting point, ending point, and trajectory of the three-dimensional line segment are used as the starting point, ending point, and insertion route of the electrode needle to obtain the insertion path information.

7. The needle guide device according to claim 1, characterized in that, The robotic arm pose planning module is also used for: The coordinate systems of the optical positioning device and the robotic arm are calibrated to obtain the first transformation matrix of the coordinate systems of the optical positioning device and the robotic arm; The coordinate systems of the 3D model and the robotic arm are calibrated to obtain the second transformation matrix between the coordinate systems of the 3D model and the robotic arm. The pre-adjustment pose of the robotic arm is obtained through the optical positioning device; The needle insertion pose of the robotic arm is obtained through the needle insertion path information and the virtual reference surface; The pose adjustment information is obtained based on the first transformation matrix, the second transformation matrix, the pose before adjustment, and the needle insertion pose.

Images