Puncture guiding method and device, medical imaging equipment and storage medium

Abstract

The invention relates to the technical field of surgical navigation, and discloses a puncture guiding method and device, medical imaging equipment and a storage medium. Firstly, a puncture path is planned based on a three-dimensional medical scanning image, and a planned needle inserting position and a planned puncture angle are determined. Then, coordinate transformation and projection transformation are carried out based on the planned needle insertion position, the planned puncture angle and the length of the puncture needle, and a projection needle insertion position and a projection needle tail position corresponding to the planned needle insertion position on a camera plane are obtained; and finally, guiding the puncture operation based on the projected needle entry position and the projected needle tail position. Through the implementation mode, the position of the bed does not need to be adjusted, so that the operation steps are simplified, the operation efficiency is improved, and the puncture guide precision is improved by reducing errors caused by equipment movement. In addition, by adopting the implementation mode, the scanning frequency is reduced, so that the radiation dose is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of surgical navigation technology, and in particular to a puncture guidance method, device, medical imaging equipment and storage medium. Background Art

[0002] Aspiration surgery is the use of a long, thin needle or instrument to penetrate the skin and underlying soft tissue for specific therapeutic or diagnostic purposes.

[0003] In the related art, a cross laser positioning light is installed at the detector position, and the C-arm angle is adjusted and the operating table is moved to ensure that the laser accurately irradiates the puncture position and guides the puncture operation. However, the movement of the equipment will introduce errors and affect the accuracy of the puncture guidance. Summary of the invention

[0004] The embodiments of this specification are intended to solve at least one of the technical problems in the related art to a certain extent. To this end, the embodiments of this specification propose a puncture guidance method, device, medical imaging equipment and storage medium.

[0005] The present specification provides a puncture guidance method, the method comprising:

[0006] Plan the puncture path based on 3D medical scanning images to determine the planned needle insertion position and planned puncture angle;

[0007] Based on the planned needle insertion position, the planned puncture angle, and the puncture needle length, coordinate transformation and projection transformation are performed to obtain a projected needle insertion position and a projected needle tail position corresponding to the planned needle insertion position on the camera plane;

[0008] The puncture operation is guided based on the projected needle entry position and the projected needle tail position.

[0009] In one embodiment, guiding the puncture operation based on the projected needle entry position and the projected needle tail position includes:

[0010] Perform target detection on the captured image to determine the real-time needle insertion position and real-time needle tail position;

[0011] The puncture operation is guided based on the relative position between the real-time needle insertion position and the projected needle insertion position, and the relative position between the real-time needle tail position and the projected needle tail position.

[0012] In one embodiment, guiding the puncture operation based on the relative position of the real-time needle insertion position and the projected needle insertion position, and the relative position of the real-time needle tail position and the projected needle tail position, includes:

[0013] A puncture operation is performed when the relative position of the real-time needle insertion position and the projected needle insertion position indicate that the real-time needle insertion position overlaps with the projected needle insertion position, and the relative position of the real-time needle tail position and the projected needle tail position indicate that the real-time needle tail position overlaps with the projected needle tail position.

[0014] In one embodiment, the three-dimensional medical scan image includes a body center position, and the puncture path planning based on the three-dimensional medical scan image to determine the planned needle insertion position and the planned puncture angle includes:

[0015] Performing puncture path planning on the three-dimensional medical scan image to determine the planned path;

[0016] Determining the planned puncture angle according to the planned path;

[0017] The planned needle insertion position is determined based on the body center position of the three-dimensional medical scan image.

[0018] In one embodiment, the coordinate transformation and projection transformation are performed based on the planned needle insertion position, the planned puncture angle, and the puncture needle length to obtain the projected needle insertion position and the projected needle tail position corresponding to the planned needle insertion position on the camera plane, including:

[0019] Determining the planned needle tail position according to the planned puncture angle and the puncture needle length;

[0020] Coordinate transformation and projection transformation are performed based on the planned needle insertion position and the planned needle tail position to obtain the projected needle insertion position and the projected needle tail position on the camera plane.

[0021] In one embodiment, the coordinate transformation and projection transformation are performed based on the planned needle insertion position and the planned needle tail position to obtain the projected needle insertion position and the projected needle tail position on the camera plane, including:

[0022] Based on the center position of the detector, determine the camera position;

[0023] Taking the camera position as the origin, coordinate transformation is performed on the planned needle insertion position and the planned needle tail position to obtain the puncture needle insertion position and the puncture needle tail position in the camera coordinate system;

[0024] The puncture needle insertion position and the puncture needle tail position are projected and transformed to obtain the projected needle insertion position and the projected needle tail position on the camera plane.

[0025] In one embodiment, the detector is located at one end of the C-arm, and determining the center position of the detector includes:

[0026] The C-arm is adjusted based on the planned puncture angle to determine the center position of the detector.

[0027] In one embodiment, the performing target detection on the captured image to determine the real-time needle insertion position and the real-time needle tail position includes:

[0028] Acquire the captured image; wherein the captured image is captured by a camera installed on a medical imaging device;

[0029] The puncture needle recognition model is used to detect the puncture needle on the captured image to determine the real-time needle insertion position and the real-time needle tail position.

[0030] In one embodiment, when the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position overlaps with the projected needle insertion position, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, performing the puncture operation includes:

[0031] When the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position and the projected needle insertion position overlap, acquiring a reference image; wherein the reference image is obtained by photographing with a camera while keeping the real-time needle insertion position unchanged;

[0032] Performing target detection on the reference image to determine the real-time needle tail position;

[0033] In a case where the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position and the projected needle tail position overlap, a puncture operation is performed.

[0034] The present specification provides a puncture guide device, the device comprising:

[0035] A planning data determination module is used to plan the puncture path based on the three-dimensional medical scanning image, and determine the planned needle insertion position and the planned puncture angle;

[0036] A projection data determination module, configured to perform coordinate transformation and projection transformation based on the planned needle insertion position, the planned puncture angle, and the puncture needle length, to obtain a projected needle insertion position and a projected needle tail position corresponding to the planned needle insertion position on the camera plane;

[0037] The puncture operation guiding module is used to guide the puncture operation based on the projected needle insertion position and the projected needle tail position.

[0038] An embodiment of the present specification provides a medical imaging device, which includes: a memory, and one or more processors communicatively connected to the memory; the memory stores instructions executable by the one or more processors, and the instructions are executed by the one or more processors to enable the one or more processors to implement the steps of the method described in any of the above embodiments.

[0039] The embodiments of this specification provide a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0040] An embodiment of the present specification provides a computer program product, wherein the computer program product includes instructions, and when the instructions are executed by a processor of a computer device, the computer device is enabled to perform the steps of the method described in any one of the above embodiments.

[0041] In the above-described implementation manner, first, the puncture path is planned based on the three-dimensional medical scanning image to determine the planned needle insertion position and the planned puncture angle. Then, coordinate transformation and projection transformation are performed based on the planned needle insertion position, the planned puncture angle, and the puncture needle length to obtain the projected needle insertion position and the projected needle tail position corresponding to the planned needle insertion position on the camera plane. Finally, the puncture operation is guided based on the projected needle insertion position and the projected needle tail position. Through the above-described implementation manner, there is no need to adjust the position of the bed, thereby simplifying the operating steps, improving surgical efficiency, and improving the accuracy of puncture guidance by reducing errors caused by equipment movement. In addition, the above-described implementation manner reduces the number of scans, thereby effectively reducing the radiation dose. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1a A schematic diagram of installing a camera in the detector portion of a DSA device provided for an embodiment of this specification;

[0043] Figure 1b A schematic diagram of the coordinates required in the puncture guidance method provided in the embodiment of this specification;

[0044] Figure 2 A schematic diagram of the process of the puncture guidance method provided in the embodiment of this specification;

[0045] Figure 3 A schematic diagram of a flow chart of a guided puncture operation provided in an embodiment of this specification;

[0046] Figure 4 A schematic diagram of a process for determining a planned needle insertion position provided in an embodiment of this specification;

[0047] Figure 5A schematic diagram of a process for obtaining a projected needle entry position and a projected needle tail position provided in an embodiment of this specification;

[0048] Figure 6 A schematic diagram of a process for obtaining a projected needle entry position and a projected needle tail position provided in an embodiment of this specification;

[0049] Figure 7 A schematic diagram of a process for determining a real-time needle insertion position and a real-time needle tail position provided in an embodiment of this specification;

[0050] Figure 8 A schematic diagram of a process for performing a puncture operation provided in an embodiment of this specification;

[0051] Fig. 9 A schematic diagram of the process of the puncture guidance method provided in the embodiment of this specification;

[0052] Fig.10 A schematic diagram of a puncture guide device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0053] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0054] A puncture procedure usually uses a long, thin needle or tool to puncture the skin and the underlying soft tissue to achieve specific therapeutic or diagnostic purposes, such as extracting tissue samples, injecting drugs, placing catheters, or performing other medical procedures. Puncture surgery is widely used in many medical fields, including radiology, oncology, interventional radiology, neurosurgery, and cardiology. The DSA (Digital Subtraction Angiography) system provides real-time, high-definition vascular images during puncture surgery, helping doctors to perform puncture operations accurately. Through DSA guidance, doctors can observe the direction of blood vessels, the location of lesions, and surrounding tissues, so as to accurately guide the puncture needle or catheter to the target area and ensure the accuracy of the operation.

[0055] Before the puncture surgery under DSA guidance begins, the doctor usually uses preoperative CT or CT-like images to plan the puncture path in detail. The doctor performs the puncture operation on the patient's body surface according to the planned puncture path. The position and accuracy of the puncture needle are monitored through DSA fluoroscopy or CT-like scanning, and the position of the puncture needle is adjusted as needed to ensure that the needle reaches the designated lesion accurately. The consistency of the puncture path with the planned path directly affects the number of adjustments and scans. In order to improve the accuracy of the puncture path, most DSA devices are equipped with a puncture navigation function to assist doctors in precise positioning.

[0056] In the related art, a cross laser positioning light is installed at the detector position to assist in positioning. First, after determining the puncture path, adjust the C-arm to an angle parallel to the planned puncture needle. Then, by moving the bed, the position of the planned puncture needle is made to coincide with the position of the laser light projection. During the guidance process, when the needle tail coincides with the cross positioning light, it means that the actual puncture path is consistent with the planned path, and puncture can begin at this time. However, the movement of the equipment will introduce errors, thereby affecting the accuracy of the puncture guidance.

[0057] In the related art, the use of real-time DSA (digital subtraction angiography) scanning technology can effectively improve accuracy. First, a real-time DSA scan is performed before puncture to obtain a perspective view, and the planned puncture needle position is marked on the image. Then, the position of the puncture needle is monitored in real time to see if it coincides with the planned puncture needle. When the planned puncture needle coincides with the puncture needle scanned in real time, the actual puncture path coincides with the planned path, and the puncture operation can be started. The real-time DSA (digital subtraction angiography) scanning method only requires adjusting the position of the C-arm, without moving the bed, reducing operational interference to patients and doctors. However, real-time perspective scanning increases the radiation dose for patients and doctors.

[0058] Based on this, the implementation method of this specification provides a puncture guidance method. First, the puncture path is planned based on the three-dimensional medical scanning image to determine the planned needle insertion position and the planned puncture angle. Then, coordinate transformation and projection transformation are performed based on the planned needle insertion position, the planned puncture angle, and the puncture needle length to obtain the projected needle insertion position and the projected needle tail position corresponding to the planned needle insertion position on the camera plane. Finally, based on the projected needle insertion position and the projected needle tail position, the puncture operation is guided. Through the above implementation, there is no need to adjust the position of the bed, thereby simplifying the operating steps and improving the surgical efficiency, and by reducing the errors caused by equipment movement, the accuracy of puncture guidance is improved. In addition, the above implementation reduces the number of scans, thereby effectively reducing the radiation dose.

[0059] The puncture guidance method provided in the embodiments of this specification can be applied to a DSA device, wherein the DSA device includes a C-arm, and the detector is located at one end of the C-arm. Figure 1a, a camera 102 is installed in the detector part of the DSA device. The DSA device is used to scan and obtain a CT-like image. Figure 1b , at this time, based on the center position P2 of the detector and the position P3 of the tube, the patient center P0 is determined. The doctor plans the puncture path on the CT-like image and determines the planned path. The center of the CT-like image is used as the body center position. It should be noted that the position corresponding to the body center position in the physical space is the patient center P0. On the CT-like image, the planned needle insertion position is determined based on the body center position of the CT-like image. According to the planned path, the planned puncture angle is determined. Then a puncture needle of appropriate length is selected, and the planned needle tail position is determined according to the planned puncture angle and the puncture needle length. The coordinate system where the planned needle insertion position and the planned needle tail position are located is converted to the world coordinate system to obtain the planned needle insertion position P5 in the world coordinate system and the planned needle tail position P4 in the world coordinate system. Among them, the positional relationship between the camera position P1 and the center position P2 of the detector is fixed.

[0060] Next, adjust the C-arm based on the planned puncture angle, and the position of the detector will also change. According to the adjusted position, determine the center position of the detector in the world coordinate system. Then, based on the center position of the detector, determine the camera position of the adjusted camera in the world coordinate system. Taking the camera position as the origin, perform coordinate transformation on the planned needle insertion position P5 in the world coordinate system and the planned needle tail position P4 in the world coordinate system to obtain the puncture needle insertion position and puncture needle tail position in the camera coordinate system. Perform projection transformation on the puncture needle insertion position and puncture needle tail position to obtain the projected needle insertion position and projected needle tail position on the camera plane.

[0061] Then, the camera is used to shoot and obtain the shot image. The puncture needle recognition model is used to detect the puncture needle on the shot image to determine the real-time needle insertion position and the real-time needle tail position. When the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position overlaps with the projected needle insertion position, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, the puncture operation is performed according to the real-time needle insertion position and the real-time needle tail position at this time, thereby realizing convenient, fast and high-precision puncture navigation.

[0062] It should be noted that the needle insertion position in the embodiments of this specification refers to the position of the needle tip of the puncture needle on the surface of the human body, rather than the position where the puncture needle has been inserted into the body.

[0063] This specification provides a puncture guidance method. Figure 2 The puncture guidance method may include the following steps:

[0064] S210: Perform puncture path planning based on the three-dimensional medical scanning image to determine the planned needle insertion position and the planned puncture angle.

[0065] Specifically, first, a medical imaging device is used to scan and obtain a three-dimensional medical scan image. Then, according to the patient's condition and specific clinical needs, the puncture path is planned on the three-dimensional medical scan image, and the planned puncture path is marked on the three-dimensional medical scan image. According to the marked planned puncture path, the planned needle insertion position and the planned puncture angle are determined. Among them, the three-dimensional medical scan image can be a CT-like image scanned by DSA.

[0066] In some embodiments, the doctor manually plans the puncture path based on the structural features displayed on the three-dimensional medical scan image and clinical experience, thereby determining the planned needle insertion position and the planned puncture angle.

[0067] In other embodiments, the path planning algorithm takes into account the location of the lesion, surrounding anatomical structures, etc., and uses the path planning algorithm to automatically plan the puncture path on the three-dimensional medical scan image, thereby determining the planned needle insertion position and the planned puncture angle.

[0068] S220, performing coordinate transformation and projection transformation based on the planned needle insertion position, the planned puncture angle, and the puncture needle length, to obtain a projected needle insertion position and a projected needle tail position corresponding to the planned needle insertion position on the camera plane.

[0069] S230: Guiding the puncture operation based on the projected needle insertion position and the projected needle tail position.

[0070] Specifically, since the needle insertion position and needle tail position obtained by the camera are displayed on the camera plane, the planned needle insertion position, planned puncture angle and puncture needle length are used to perform coordinate transformation and projection transformation, and the planned needle insertion position and planned needle tail position are transformed to the camera plane corresponding to the camera, and the projected needle insertion position and projected needle tail position on the camera plane are obtained, so that the needle insertion position and needle tail position obtained by subsequent camera photography and the projected needle tail position are displayed on the same plane for comparison to ensure the accuracy of puncture. Then the projected needle insertion position and projected needle tail position are matched with the needle insertion position and needle tail position obtained by the camera photography, and based on the matching results fed back in real time, the doctor is guided to adjust the needle insertion position and angle of the puncture needle to correct the puncture deviation and guide the puncture operation.

[0071] In the above embodiment, first, the puncture path is planned based on the three-dimensional medical scanning image to determine the planned needle insertion position and the planned puncture angle. Then, coordinate transformation and projection transformation are performed based on the planned needle insertion position, the planned puncture angle, and the puncture needle length to obtain the projected needle insertion position and the projected needle tail position corresponding to the planned needle insertion position on the camera plane. Finally, the puncture operation is guided based on the projected needle insertion position and the projected needle tail position. Through the above embodiment, there is no need to adjust the position of the bed, thereby simplifying the operating steps and improving the surgical efficiency, and by reducing the error caused by the movement of the equipment, the accuracy of the puncture guidance is improved. In addition, the above embodiment reduces the number of scans, thereby effectively reducing the radiation dose.

[0072] In some embodiments, see Figure 3 Based on the projected needle insertion position and the projected needle tail position, guiding the puncture operation may include the following steps:

[0073] S310, performing target detection on the captured image to determine the real-time needle insertion position and the real-time needle tail position.

[0074] S320: guiding the puncture operation based on the relative position between the real-time needle insertion position and the projected needle insertion position, and the relative position between the real-time needle tail position and the projected needle tail position.

[0075] Specifically, a visual recognition method is used, such as a target recognition model, to detect the target in the captured image, analyze the features in the captured image, and automatically identify the puncture needle in the captured image, thereby determining the real-time needle insertion position and the real-time needle tail position. The relative position of the real-time needle insertion position and the projected needle insertion position, as well as the relative position of the real-time needle tail position and the projected needle tail position are matched, and based on the matching results fed back in real time, the doctor is guided to adjust the needle insertion position and angle of the puncture needle to correct the puncture deviation and guide the puncture operation.

[0076] In the above embodiment, target detection is performed on the captured image to determine the real-time needle insertion position and the real-time needle tail position. Based on the relative position of the real-time needle insertion position and the projected needle insertion position, as well as the relative position of the real-time needle tail position and the projected needle tail position, the puncture operation is guided to ensure that the final puncture position overlaps with the planned puncture position, thereby achieving a high-precision puncture process.

[0077] In some embodiments, guiding the puncture operation based on the relative position of the real-time needle insertion position and the projected needle insertion position, as well as the relative position of the real-time needle tail position and the projected needle tail position, may include: performing the puncture operation when the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position overlaps with the projected needle insertion position, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position.

[0078] Specifically, before performing the puncture operation, the target recognition model is used to perform target detection on the captured image, analyze the features in the captured image, and automatically identify the puncture needle in the captured image, thereby determining the real-time needle insertion position and the real-time needle tail position. Subsequently, the real-time needle insertion position is compared with the projected needle insertion position, and the real-time needle tail position is compared with the projected needle tail position. By analyzing their relative positions, it is determined whether they overlap, thereby ensuring the accuracy of the puncture. When the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position overlaps with the projected needle insertion position, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, the puncture operation is performed. At this time, the puncture path formed by the needle tip position and the inclination angle of the puncture needle is highly consistent with the planned puncture path, thereby ensuring the accuracy of the puncture path of the puncture operation.

[0079] It should be noted that the projected needle insertion position and the projected needle tail position are used as a reference and displayed on the display of the medical imaging device, providing doctors with necessary visual references. The real-time needle insertion position and the real-time needle tail position will be displayed in real time on the display of the medical imaging device, ensuring that doctors can monitor the current state and position of the puncture needle at any time to determine the positional relationship between the real-time needle insertion position and the real-time needle tail position and the projected needle insertion position and the projected needle tail position, helping doctors to adjust the puncture needle's insertion position and tilt angle in time.

[0080] In some embodiments, the real-time needle insertion position is compared with the projected needle insertion position to analyze the relative error between the two. Then, it is determined whether the relative error is within a preset acceptable range to ensure the puncture accuracy. When the error between the real-time needle insertion position and the projected needle insertion position is within the allowable range, it is considered that the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position and the projected needle insertion position overlap.

[0081] The real-time needle tail position is compared with the projected needle tail position, and the relative error between the two is analyzed. Then, it is determined whether the relative error is within the preset acceptable range to ensure the puncture accuracy. When the error between the real-time needle tail position and the projected needle tail position is within the allowable range, it is considered that the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position and the projected needle tail position overlap.

[0082] In the above embodiment, when the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position overlaps with the projected needle insertion position, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, the puncture operation is performed to ensure that the final puncture position overlaps with the planned puncture position, thereby achieving a high-precision puncture process.

[0083] In some embodiments, see Figure 4The three-dimensional medical scanning image includes the body center position. The puncture path planning based on the three-dimensional medical scanning image to determine the planned needle insertion position and the planned puncture angle may include the following steps:

[0084] S410: Perform puncture path planning on the three-dimensional medical scan image to determine the planned path.

[0085] S420: Determine the planned puncture angle according to the planned path.

[0086] S430: Determine the planned needle insertion position based on the body center position of the three-dimensional medical scan image.

[0087] Specifically, according to the specific location of the lesion and the actual situation, etc., the puncture path planning is performed in the three-dimensional medical scanning image, and the appropriate path is determined to obtain the planned path. The needle insertion position is determined according to the planned path, and then the needle insertion position is used as the starting point to draw a line segment parallel to the bed and calculate the angle between this line segment and the planned path, and the angle is used as the planned puncture angle. Based on the three-dimensional medical scanning image obtained by scanning with a medical imaging device, the center position of the three-dimensional medical scanning image is determined to obtain the body center position. In the three-dimensional medical scanning image, according to the positional relationship between the intersection between the planned path and the patient and the body center position, the body center position is used to determine the position of the intersection between the planned path and the patient in the three-dimensional medical scanning image, and use it as the planned needle insertion position. It should be noted that the body center position can be the patient center.

[0088] In the above implementation, the puncture path is planned for the three-dimensional medical scanning image, the planned path is determined, the planned puncture angle is determined based on the planned path, and the planned needle insertion position is determined based on the body center position of the three-dimensional medical scanning image, which helps to improve the accuracy of puncture and reduce errors in actual operation.

[0089] In some embodiments, see Figure 5 , coordinate transformation and projection transformation are performed based on the planned needle insertion position, the planned puncture angle, and the puncture needle length to obtain the projected needle insertion position and the projected needle tail position corresponding to the planned needle insertion position on the camera plane, which may include the following steps:

[0090] S510: Determine the planned needle tail position according to the planned puncture angle and the puncture needle length.

[0091] S520, performing coordinate transformation and projection transformation based on the planned needle entry position and the planned needle tail position to obtain the projected needle entry position and the projected needle tail position on the camera plane.

[0092] Specifically, a puncture needle of appropriate length is selected according to the specific conditions of the patient and clinical needs. After determining the required puncture needle length, the proportional coefficient between the image and the actual situation is calculated based on the comparison between the reference object in the three-dimensional medical scan image and its actual length. Using the proportional coefficient, according to the calculated proportional coefficient, the actual puncture needle length is converted into the corresponding puncture needle length in the three-dimensional medical scan image. Then, in the three-dimensional medical scan image, the corresponding puncture needle length in the three-dimensional medical scan image is taken as the starting point of the planned needle insertion position, the planned puncture angle is tilted, and the planned needle tail position is determined by the direction vector corresponding to the corresponding puncture needle length in the three-dimensional medical scan image and the planned puncture angle. Since the needle insertion position and needle tail position obtained by the camera are displayed on the camera plane, the determined planned needle insertion position and planned needle tail position are used for coordinate conversion and projection conversion, and the planned needle insertion position and planned needle tail position are converted to the camera plane corresponding to the camera, and the projected needle insertion position and projected needle tail position on the camera plane are obtained, so that the needle insertion position and needle tail position obtained by the subsequent camera shooting are compared with the projected needle insertion position and projected needle tail position displayed on the same plane to ensure the accuracy of the puncture.

[0093] In the above embodiment, the planned needle tail position is determined according to the planned puncture angle and the puncture needle length, and coordinate transformation and projection transformation are performed based on the planned needle insertion position and the planned needle tail position to obtain the projected needle insertion position and the projected needle tail position on the camera plane, which helps to improve the accuracy of puncture and reduce errors in actual operation.

[0094] In some embodiments, see Figure 6 , coordinate transformation and projection transformation are performed based on the planned needle insertion position and the planned needle tail position to obtain the projected needle insertion position and the projected needle tail position on the camera plane, which may include the following steps:

[0095] S610: Determine the camera position based on the center position of the detector.

[0096] S620, taking the camera position as the origin, performing coordinate transformation on the planned needle insertion position and the planned needle tail position, and obtaining the puncture needle insertion position and the puncture needle tail position in the camera coordinate system.

[0097] S630, performing projection conversion on the puncture needle insertion position and the puncture needle tail position to obtain the projected needle insertion position and the projected needle tail position on the camera plane.

[0098] Specifically, after adjusting the C-arm, the center position of the detector at this time is determined based on the built-in coordinate system of the medical imaging device. Then, according to the spatial relationship and geometric configuration between the camera and the detector, the center position of the detector is used to determine the camera position. The camera position is converted from the coordinate system of the medical imaging device to the world coordinate system, and the position of the camera in the world coordinate system is determined, thereby obtaining the camera position in the world coordinate system. The planned needle insertion position and the planned needle tail position are converted from the image coordinate system to the world coordinate system to obtain the planned needle insertion position and the planned needle tail position in the world coordinate system. Then, taking the camera position in the world coordinate system as the origin, the planned needle insertion position and the planned needle tail position in the world coordinate system are transformed to obtain the puncture needle insertion position and the puncture needle tail position in the camera coordinate system. Then, using the camera's internal reference matrix, the puncture needle insertion position and the puncture needle tail position are projected and transformed to obtain the projected needle insertion position and the projected needle tail position on the camera plane, thereby drawing the planned puncture needle on the camera plane. The projected needle insertion position and the projected needle tail position serve as the benchmark for subsequent real-time matching to ensure the accuracy of the needle insertion point and the needle tail point in the actual puncture operation. Among them, the camera coordinate system can be understood as the coordinate system of the camera's field of view.

[0099] In some embodiments, the medical imaging device is equipped with a camera when it leaves the factory, and the spatial relationship and geometric configuration between the center of the detector and the camera are set at the factory and fixed in the medical imaging device.

[0100] In other embodiments, the medical imaging device is not equipped with a camera, in which case the camera needs to be subsequently installed on the medical imaging device, and the spatial relationship and geometric configuration between the detector center and the camera need to be determined through a calibration process.

[0101] In the above embodiment, the camera position is determined based on the center position of the detector, and the planned needle insertion position and the planned needle tail position are transformed with the camera position as the origin to obtain the puncture needle insertion position and the puncture needle tail position in the camera coordinate system. The puncture needle insertion position and the puncture needle tail position are projected and transformed to obtain the projected needle insertion position and the projected needle tail position on the camera plane, so that the actual puncture position can be intuitively compared in actual operation.

[0102] In some embodiments, the detector is located at one end of a C-arm, and determining the center position of the detector may include: adjusting the C-arm based on a planned puncture angle to determine the center position of the detector.

[0103] Specifically, the C-arm is adjusted according to the planned puncture angle so that the angle between the center point of the detector at one end of the C-arm and the tube is consistent with the planned puncture angle, so that the line between the center point of the detector and the tube is parallel to the planned path. In some embodiments, after the puncture path for performing the puncture is determined, it is necessary to use a medical imaging device for further verification. The angle of the detector determined by adjusting the C-arm based on the planned puncture angle can be accurately aligned with the preset puncture area. The medical imaging device is used to scan again to determine whether the puncture path can act on the lesion, thereby determining whether the puncture path for performing the puncture is accurate.

[0104] After adjusting the C-arm based on the planned puncture angle, the position of the center of the detector in the coordinate system of the medical imaging device is determined based on the built-in coordinate system of the medical imaging device to obtain the center position of the detector. For example, the coordinate system of the medical imaging device usually uses the geometric center of the device as a reference point.

[0105] In the above implementation, the C-arm is adjusted based on the planned puncture angle to determine the center position of the detector, so that the camera, the planned needle insertion position, and the planned needle tail position can be subsequently converted to the same coordinate system to determine the relative positions.

[0106] In some embodiments, see Figure 7 , performing target detection on the captured image and determining the real-time needle insertion position and the real-time needle tail position may include the following steps:

[0107] S710: Acquire a captured image.

[0108] S720: Use the puncture needle recognition model to detect the puncture needle in the captured image to determine the real-time needle insertion position and the real-time needle tail position.

[0109] The captured image is obtained by a camera installed on the medical imaging device.

[0110] Specifically, first, a camera installed on a medical imaging device is used to shoot and obtain a captured image. Then, a puncture needle recognition model is used to detect the captured image, analyze the features in the captured image, and automatically identify the puncture needle in the captured image, thereby determining the real-time needle insertion position and the real-time needle tail position. The puncture needle recognition model may be a target detection model determined based on a convolutional neural network (CNN) or other deep learning technologies, such as a YOLO (You Only Look Once) network or a Faster R-CNN network.

[0111] Exemplarily, in order to build a high-quality sample set, a large number of photos of puncture needles should be collected. The sample set should cover different angles (such as front, side, and top view) to fully demonstrate the appearance of the puncture needle. The sample set can also include different positions of the puncture needle, such as the needle head and the needle tail, and cover puncture needles of various lengths, such as short needles, medium-length needles, and long needles. The shooting should be carried out under a variety of background information, such as different types of medical imaging equipment (such as ultrasound, CT, and MRI equipment) to simulate different scenarios in actual use. In addition, a variety of lighting conditions should be recorded, such as natural light, artificial light sources of different intensities, and shadow effects, to ensure that the performance of the puncture needle in various lighting environments is captured. Through this diverse sample set, the characteristics of the puncture needle can be deeply studied, and more comprehensive feature extraction and more accurate model training can be achieved. The sample set is annotated with puncture needles to determine the label of the sample set. For example, the puncture needles can be annotated manually.

[0112] The sample set is input into the initial puncture needle recognition model to extract the features of the puncture needle, and a training puncture needle can be obtained. Then, the loss value of the initial puncture needle recognition model can be determined based on the label corresponding to the sample set and the training puncture needle, and the parameters of the initial puncture needle recognition model are updated based on the model loss value. Similarly, the updated initial puncture needle recognition model is continued to be trained, and when the model training stop condition is reached, a trained puncture needle recognition model can be obtained. The model training stop condition can be that the model loss value tends to converge, or that the number of training rounds reaches a preset number of rounds.

[0113] Please note that Figure 1a The camera 102 can be installed at one end of the C-arm of the DSA device. The C-arm is adjusted based on the planned puncture angle to maintain an appropriate viewing angle between the camera and the puncture area, which helps to reduce image distortion that may be caused by viewing angle deviation, thereby reducing diagnostic errors. At this time, the camera can cover the approximate area where the patient needs to be punctured, so as to obtain the captured image and reference image for detecting the puncture needle.

[0114] In the above implementation, a captured image is acquired, and a puncture needle recognition model is used to detect the puncture needle on the captured image to determine the real-time needle insertion position and the real-time needle tail position, thereby providing a basis for subsequently determining the position for performing puncture.

[0115] In some embodiments, see Figure 8 In the case where the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position and the projected needle insertion position overlap, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position and the projected needle tail position overlap, performing the puncture operation may include the following steps:

[0116] S810. Acquire a reference image when the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position and the projected needle insertion position overlap.

[0117] The reference image is obtained by taking pictures with a camera while keeping the real-time needle insertion position unchanged.

[0118] Specifically, before puncture, the captured image is subjected to target detection through the target recognition model, the features in the captured image are analyzed, and the puncture needle in the captured image is automatically identified, thereby determining the real-time needle insertion position. Subsequently, the real-time needle insertion position is compared with the projected needle insertion position, and by analyzing the relative positions of the two, it is determined whether they overlap, thereby ensuring the accuracy of the puncture. When the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position overlaps with the projected needle insertion position, in order to further verify and ensure the accuracy of the puncture, it is necessary to obtain a reference image by photographing with a camera while keeping the real-time needle insertion position unchanged. Exemplarily, the puncture needle recognition model is used to perform puncture needle detection on the captured image, the features in the captured image are analyzed, and the puncture needle in the captured image is automatically identified, thereby determining the real-time needle insertion position.

[0119] In some embodiments, the real-time needle insertion position is compared with the projected needle insertion position to analyze the relative error between the two. Then, it is determined whether the relative error is within a preset acceptable range to ensure the puncture accuracy. When the error between the real-time needle insertion position and the projected needle insertion position is within the allowable range, it is considered that the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position and the projected needle insertion position overlap.

[0120] S820, performing target detection on the reference image to determine the real-time needle tail position.

[0121] S830: When the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position and the projected needle tail position overlap, perform a puncture operation.

[0122] Specifically, after acquiring the reference image, the reference image is subjected to target detection through the target recognition model, the features in the reference image are analyzed, and the puncture needle in the captured image is automatically identified, thereby determining the real-time needle tail position. Subsequently, the real-time needle tail position is compared with the projected needle tail position, and by analyzing the relative positions of the two, it is determined whether they overlap, thereby ensuring the accuracy of the puncture. When it is determined that the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, the puncture operation is performed. At this time, the puncture path formed by the needle tip position and the inclination angle of the puncture needle is highly consistent with the planned puncture path, thereby ensuring the accuracy of the puncture path of the puncture operation. Exemplarily, the puncture needle recognition model is used to perform puncture needle detection on the reference image, analyze the features in the reference image, and automatically identify the puncture needle in the reference image, thereby determining the real-time needle tail position.

[0123] In the above embodiment, when the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position and the projected needle insertion position overlap, a reference image is acquired, target detection is performed on the reference image, and the real-time needle tail position is determined. When the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position and the projected needle tail position overlap, the puncture operation is performed to ensure that the final puncture position overlaps with the planned puncture position, thereby achieving a high-precision puncture process.

[0124] The present specification also provides a puncture guidance method. The three-dimensional medical scanning image includes the center position of the body, and the detector is located at one end of the C-arm. For example, see Fig. 9 The puncture guidance method may include the following steps:

[0125] S902: Perform puncture path planning on the three-dimensional medical scan image to determine the planned path.

[0126] S904: Determine the planned puncture angle according to the planned path.

[0127] S906: Determine the planned needle insertion position based on the body center position of the three-dimensional medical scan image.

[0128] S908. Determine the planned needle tail position according to the planned puncture angle and puncture needle length.

[0129] S910, adjusting the C-arm based on the planned puncture angle, and determining the center position of the detector.

[0130] S912. Determine the camera position based on the center position of the detector.

[0131] S914. Taking the camera position as the origin, coordinate transformation is performed on the planned needle insertion position and the planned needle tail position to obtain the puncture needle insertion position and the puncture needle tail position in the camera coordinate system.

[0132] S916, performing projection conversion on the puncture needle insertion position and the puncture needle tail position to obtain the projected needle insertion position and the projected needle tail position on the camera plane.

[0133] S918: Acquire the captured image.

[0134] The captured images are obtained by using a camera installed on the medical imaging equipment.

[0135] S920: Use the puncture needle recognition model to detect the puncture needle in the captured image to determine the real-time needle insertion position and the real-time needle tail position.

[0136] S922. When the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position and the projected needle insertion position overlap, obtain a reference image.

[0137] The reference image is obtained by taking pictures with a camera while keeping the real-time needle insertion position unchanged.

[0138] S924, perform target detection on the reference image to determine the real-time needle tail position.

[0139] S926: When the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, the puncture operation is performed.

[0140] This specification provides a puncture guide device 1000, see Fig.10 The puncture guidance device 1000 includes: a planning data determination module 1010 , a projection data determination module 1020 , and a puncture operation guidance module 1030 .

[0141] The planning data determination module 1010 is used to plan the puncture path based on the three-dimensional medical scanning image, and determine the planned needle insertion position and the planned puncture angle;

[0142] The projection data determination module 1020 is used to perform coordinate transformation and projection transformation based on the planned needle insertion position, the planned puncture angle and the puncture needle length to obtain a projected needle insertion position and a projected needle tail position corresponding to the planned needle insertion position on the camera plane;

[0143] The puncture operation guiding module 1030 is used to guide the puncture operation based on the projected needle insertion position and the projected needle tail position.

[0144] For a detailed description of the puncture guiding device, please refer to the description of the puncture guiding method above, which will not be repeated here.

[0145] In some embodiments, a medical imaging device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the method steps in the above embodiments when executing the computer program.

[0146] An embodiment of the present specification provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method in any of the above embodiments are implemented.

[0147] One embodiment of the present specification provides a computer program product, which includes instructions. When the instructions are executed by a processor of a computer device, the computer device can perform the steps of the method of any of the above embodiments.

[0148] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

Claims

1. A puncture guidance method, characterized in that: The method comprises: Plan the puncture path based on 3D medical scanning images to determine the planned needle insertion position and planned puncture angle; Based on the planned needle insertion position, the planned puncture angle, and the puncture needle length, coordinate transformation and projection transformation are performed to obtain a projected needle insertion position and a projected needle tail position corresponding to the planned needle insertion position on the camera plane; The puncture operation is guided based on the projected needle entry position and the projected needle tail position.

2. The method according to claim 1, characterized in that The step of guiding the puncture operation based on the projected needle insertion position and the projected needle tail position includes: Perform target detection on the captured image to determine the real-time needle insertion position and real-time needle tail position; The puncture operation is guided based on the relative position between the real-time needle entry position and the projected needle entry position, and the relative position between the real-time needle tail position and the projected needle tail position.

3. The method according to claim 2, characterized in that The guiding of the puncture operation based on the relative position between the real-time needle insertion position and the projected needle insertion position, and the relative position between the real-time needle tail position and the projected needle tail position, comprises: A puncture operation is performed when the relative position of the real-time needle insertion position and the projected needle insertion position indicate that the real-time needle insertion position overlaps with the projected needle insertion position, and the relative position of the real-time needle tail position and the projected needle tail position indicate that the real-time needle tail position overlaps with the projected needle tail position.

4. The method according to claim 1, characterized in that: The three-dimensional medical scan image includes the body center position, and the puncture path planning based on the three-dimensional medical scan image to determine the planned needle insertion position and the planned puncture angle includes: Performing puncture path planning on the three-dimensional medical scan image to determine the planned path; Determining the planned puncture angle according to the planned path; The planned needle insertion position is determined based on the body center position of the three-dimensional medical scan image.

5. The method according to claim 1, characterized in that The coordinate transformation and projection transformation are performed based on the planned needle insertion position, the planned puncture angle, and the puncture needle length to obtain the projected needle insertion position and the projected needle tail position corresponding to the planned needle insertion position on the camera plane, including: Determining the planned needle tail position according to the planned puncture angle and the puncture needle length; Coordinate transformation and projection transformation are performed based on the planned needle insertion position and the planned needle tail position to obtain the projected needle insertion position and the projected needle tail position on the camera plane.

6. The method according to claim 5, characterized in that The coordinate transformation and projection transformation are performed based on the planned needle insertion position and the planned needle tail position to obtain the projected needle insertion position and the projected needle tail position on the camera plane, including: Based on the center position of the detector, determine the camera position; Taking the camera position as the origin, coordinate transformation is performed on the planned needle insertion position and the planned needle tail position to obtain the puncture needle insertion position and the puncture needle tail position in the camera coordinate system; The puncture needle insertion position and the puncture needle tail position are projected and transformed to obtain the projected needle insertion position and the projected needle tail position on the camera plane.

7. The method according to claim 6, characterized in that The detector is located at one end of the C-arm, and determining the center position of the detector includes: The C-arm is adjusted based on the planned puncture angle to determine the center position of the detector.

8. The method according to claim 2, characterized in that: The target detection is performed on the captured image to determine the real-time needle insertion position and the real-time needle tail position, including: Acquire the captured image; wherein the captured image is captured by a camera installed on a medical imaging device; The puncture needle recognition model is used to detect the puncture needle on the captured image to determine the real-time needle insertion position and the real-time needle tail position.

9. The method according to claim 3, characterized in that: The method of performing a puncture operation when the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position and the projected needle insertion position overlap, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position and the projected needle tail position overlap, comprises: When the relative position of the real-time needle insertion position and the projected needle insertion position indicates that the real-time needle insertion position and the projected needle insertion position overlap, acquiring a reference image; wherein the reference image is obtained by photographing with a camera while keeping the real-time needle insertion position unchanged; Performing target detection on the reference image to determine the real-time needle tail position; In a case where the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position and the projected needle tail position overlap, a puncture operation is performed.

10. A puncture guide device, characterized in that: The device comprises: A planning data determination module is used to plan the puncture path based on the three-dimensional medical scanning image, and determine the planned needle insertion position and the planned puncture angle; A projection data determination module, configured to perform coordinate transformation and projection transformation based on the planned needle insertion position, the planned puncture angle, and the puncture needle length, to obtain a projected needle insertion position and a projected needle tail position corresponding to the planned needle insertion position on the camera plane; The puncture operation guiding module is used to guide the puncture operation based on the projected needle insertion position and the projected needle tail position.

11. A medical imaging device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images