Puncture guidance method, apparatus, medical imaging device, and storage medium

Abstract

The present application relates to the technical field of surgical navigation, and discloses a puncture guiding method and device, medical imaging equipment and a storage medium. First, a puncture path is planned based on a three-dimensional medical scan image, and a planned needle insertion position and a planned puncture angle are determined. Then, coordinate conversion and projection conversion are performed based on the planned needle insertion position, the planned puncture angle and the length of the puncture needle, to obtain a projected needle insertion position and a projected needle tail position on the camera plane corresponding to the planned needle insertion position. Finally, the puncture operation is guided based on the projected needle insertion position and the projected needle tail position. Through the above implementation, the position of the bed does not need to be adjusted, thereby simplifying the operation steps, improving the operation efficiency, and improving the accuracy of the puncture guiding by reducing the error caused by the movement of the equipment. In addition, the above implementation reduces the number of scans, thereby effectively reducing the radiation dose.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of surgical navigation, and in particular to a puncture guiding method and device, a medical imaging device, and a storage medium. BACKGROUND

[0002] Puncture surgery usually uses an elongated needle or tool to puncture the skin and underlying soft tissue to achieve a specific treatment or diagnostic purpose.

[0003] In the related art, a cross laser positioning lamp is installed at the position of the detector, and the C-arm angle is adjusted and the operating table is moved to ensure that the laser accurately irradiates the puncture position to guide the puncture operation. However, the movement of the device introduces errors, affecting the accuracy of the puncture guidance. SUMMARY

[0004] The embodiments of the present application aim to at least solve one of the technical problems in the related art. To this end, the embodiments of the present application propose a puncture guiding method and device, a medical imaging device, and a storage medium.

[0005] The embodiments of the present application provide a puncture guiding method, which comprises:

[0006] planning a puncture path based on a three-dimensional medical scan image, determining a planned needle entry position and a planned puncture angle;

[0007] performing coordinate conversion and projection conversion based on the planned needle entry position, the planned puncture angle, and the length of the puncture needle, to obtain a projected needle entry position and a projected needle tail position on the camera plane corresponding to the planned needle entry position;

[0008] guiding the puncture operation based on the projected needle entry position and the projected needle tail position.

[0009] In one of the embodiments, the guiding the puncture operation based on the projected needle entry position and the projected needle tail position comprises:

[0010] performing target detection on the captured image to determine a real-time needle entry position and a real-time needle tail position;

[0011] guiding the puncture operation based on the relative positions of the real-time needle entry position and the projected needle entry position, and the relative positions of the real-time needle tail position and the projected needle tail position.

[0012] In one of the embodiments, the guiding the puncture operation based on the relative positions of the real-time needle entry position and the projected needle entry position, and the relative positions of the real-time needle tail position and the projected needle tail position comprises:

[0013] In a case where the relative position of the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position overlaps with the projected needle entry position, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, a puncture operation is performed.

[0014] In one embodiment, the three-dimensional medical scan image includes a body center position, and the puncture path planning based on the three-dimensional medical scan image includes determining a planned needle entry position and a planned puncture angle, which includes:

[0015] The three-dimensional medical scan image is subjected to puncture path planning to determine a planned path.

[0016] According to the planned path, the planned puncture angle is determined.

[0017] Based on the body center position of the three-dimensional medical scan image, the planned needle entry position is determined.

[0018] In one embodiment, the coordinate conversion and projection conversion based on the planned needle entry position, the planned puncture angle, and the length of the puncture needle to obtain the projected needle entry position and the projected needle tail position corresponding to the planned needle entry position on the camera plane include:

[0019] According to the planned puncture angle and the length of the puncture needle, a planned needle tail position is determined.

[0020] Based on the planned needle entry position and the planned needle tail position, coordinate conversion and projection conversion are performed to obtain the projected needle entry position and the projected needle tail position on the camera plane.

[0021] In one embodiment, the coordinate conversion and projection conversion based on the planned needle entry position and the planned needle tail position to obtain the projected needle entry position and the projected needle tail position on the camera plane include:

[0022] Based on the detector center position, a camera position is determined.

[0023] With the camera position as the origin, coordinate conversion is performed on the planned needle entry position and the planned needle tail position to obtain a puncture needle entry position and a puncture needle tail position in the camera coordinate system.

[0024] The puncture needle entry position and the puncture needle tail position are subjected to projection conversion to obtain the projected needle entry position and the projected needle tail position on the camera plane.

[0025] In one embodiment, the detector is located at one end of a C-arm, and the detector center position is determined, which includes:

[0026] adjust the C-arm based on the planned puncture angle, and determine the detector center position.

[0027] In one of the embodiments, the target detection on the photographed image, the determination of the real-time needle entry position and the real-time needle tail position, comprises:

[0028] obtaining the photographed image; wherein the photographed image is obtained by a camera installed on a medical imaging device;

[0029] detecting the puncture needle on the photographed image by using a puncture needle recognition model, and determining the real-time needle entry position and the real-time needle tail position.

[0030] In one of the embodiments, in the case that the relative position between the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position overlaps with the projected needle entry position, and the relative position between the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, performing the puncture operation comprises:

[0031] In the case that the relative position between the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position overlaps with the projected needle entry position, obtaining a reference image; wherein the reference image is obtained by the camera while keeping the real-time needle entry position unchanged;

[0032] detecting the target on the reference image, and determining the real-time needle tail position;

[0033] In the case that the relative position between the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, performing the puncture operation.

[0034] The embodiments of the present specification provide a puncture guiding device, which comprises:

[0035] a planning data determination module for determining a planned needle entry position and a planned puncture angle based on three-dimensional medical scan images;

[0036] a projection data determination module for performing coordinate conversion and projection conversion based on the planned needle entry position, the planned puncture angle and the length of the puncture needle, to obtain a projected needle entry position and a projected needle tail position corresponding to the planned needle entry position on a camera plane;

[0037] a puncture operation guiding module for guiding the puncture operation based on the projected needle entry position and the projected needle tail position.

[0038] The medical imaging device provided by the embodiments of the present specification includes a memory and one or more processors connected in communication with the memory; the memory stores instructions executable by the one or more processors, and the instructions are executed by the one or more processors to enable the one or more processors to implement the steps of the method according to any one of the embodiments described above.

[0039] The embodiments of the present specification provide a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method according to any one of the embodiments described above.

[0040] The embodiments of the present specification provide a computer program product, which includes instructions, and the instructions are executed by a processor of a computer device to enable the computer device to perform the steps of the method according to any one of the embodiments described above.

[0041] In the above embodiments of the present specification, first, a puncture path is planned based on a three-dimensional medical scan image to determine a planned needle insertion position and a planned puncture angle. Then, based on the planned needle insertion position, the planned puncture angle, and the length of the puncture needle, coordinate conversion and projection conversion are performed to obtain a projected needle insertion position and a projected needle tail position on the camera plane corresponding to the planned needle insertion position. Finally, based on the projected needle insertion position and the projected needle tail position, a puncture operation is guided. Through the above embodiments, the position of the bed does not need to be adjusted, thereby simplifying the operation steps, improving the operation efficiency, and improving the accuracy of the puncture guidance by reducing the errors caused by the movement of the equipment. In addition, the above embodiments reduce the number of scans, thereby effectively reducing the radiation dose. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1a A schematic diagram of installing a camera on the detector part of the DSA device is provided for the embodiments of the present specification;

[0043] Figure 1b A schematic diagram of the required coordinates in the puncture guidance method is provided for the embodiments of the present specification;

[0044] Figure 2 A flowchart of the puncture guidance method is provided for the embodiments of the present specification;

[0045] Figure 3 A flowchart of guiding the puncture operation is provided for the embodiments of the present specification;

[0046] Figure 4 A flowchart of determining the planned needle insertion position is provided for the embodiments of the present specification;

[0047] Figure 5A flowchart for obtaining a projected needle entry position and a projected needle tail position is provided for the embodiments of the present specification;

[0048] Figure 6 A flowchart for obtaining a projected needle entry position and a projected needle tail position is provided for the embodiments of the present specification;

[0049] Figure 7 A flowchart for determining a real-time needle entry position and a real-time needle tail position is provided for the embodiments of the present specification;

[0050] Figure 8 A flowchart for performing a puncture operation is provided for the embodiments of the present specification;

[0051] Figure 9 A flowchart for a puncture guiding method is provided for the embodiments of the present specification;

[0052] Figure 10 A schematic diagram of a puncture guiding device is provided for the embodiments of the present specification. DETAILED DESCRIPTION

[0053] Embodiments of the present application are described in detail below with reference to the attached drawing figures, wherein the same or like reference numerals and characters throughout the figures denote the same or like components or elements having the same or similar functions. The embodiments described below are exemplary and are intended to be illustrative of the present application and are not to be construed as limiting thereof.

[0054] Puncture procedures generally use an elongated needle or tool to puncture the skin and underlying soft tissues for specific therapeutic or diagnostic purposes, such as extracting tissue samples, injecting drugs, placing catheters, or performing other medical operations. Puncture procedures are widely used in many medical fields, including radiology, oncology, interventional radiology, neurosurgery, and cardiology. DSA (Digital Subtraction Angiography) systems provide real-time, high-definition vascular images during puncture procedures, helping doctors to perform puncture operations accurately. Through DSA guidance, doctors can observe the blood vessel path, lesion location, and surrounding tissues, thereby accurately guiding the puncture needle or catheter to the target area, ensuring the accuracy of the operation.

[0055] Before a puncture operation under the guidance of DSA, the doctor usually plans the puncture path in detail by using the preoperative CT or CT-like image. The doctor punctures the patient's body surface according to the planned puncture path. The position of the puncture needle and its accuracy are monitored by DSA perspective or CT-like scanning, and the position of the puncture needle is adjusted as needed to ensure that the needle head accurately reaches the specified lesion. The consistency of the puncture path with the planned path directly affects the number of adjustments and scans. In order to improve the accuracy of the puncture path, most DSA devices are equipped with puncture navigation functions to assist the doctor in accurate positioning.

[0056] In the related art, a cross laser positioning lamp is installed at the position of the detector to assist positioning. First, after the puncture path is determined, the C-arm is adjusted to an angle parallel to the planned puncture needle. Then, by moving the bed, the position of the planned puncture needle is made to coincide with the position of the laser lamp. During the guidance process, when the needle tail coincides with the cross positioning lamp, it means that the actual puncture path is consistent with the planned path, and at this time the puncture can be started. However, the movement of the device introduces errors, thereby affecting the accuracy of the puncture guidance.

[0057] In the related art, the use of real-time DSA (digital subtraction angiography) scanning technology can effectively improve the accuracy. First, a real-time DSA scan is performed before puncture to obtain a perspective view, and the planned puncture needle position is marked on the image. Then, the position of the puncture needle is monitored in real time to coincide with the planned puncture needle. When the planned puncture needle coincides with the real-time scanned puncture needle, the actual puncture path coincides with the planned path, and the puncture operation can be started. The real-time DSA (digital subtraction angiography) scanning method only needs to adjust the position of the C-arm, without moving the patient bed, reducing the operational interference to the patient and the doctor. However, real-time perspective scanning increases the radiation dose of the patient and the doctor.

[0058] Based on this, the present specification embodiment provides a puncture guidance method. First, a puncture path is planned based on a three-dimensional medical scan image, and a planned needle entry position and a planned puncture angle are determined. Then, based on the planned needle entry position, the planned puncture angle, and the length of the puncture needle, coordinate conversion and projection conversion are performed to obtain a projected needle entry position and a projected needle tail position corresponding to the planned needle entry position on the camera plane. Finally, based on the projected needle entry position and the projected needle tail position, the puncture operation is guided. Through the above embodiment, the position of the bed does not need to be adjusted, thereby simplifying the operation steps, improving the operation efficiency, and improving the accuracy of the puncture guidance by reducing the errors caused by the movement of the device. In addition, the above embodiment reduces the number of scans, thereby effectively reducing the radiation dose.

[0059] The puncture guidance method provided by the present specification embodiment can be applied to a DSA device, which includes a C-arm, and a detector located at one end of the C-arm. Please refer to Figure 1aA camera 102 is installed in the detector part of the DSA device. A CT-like image is obtained by scanning through the DSA device. Please refer to Figure 1b At this time, the patient center P0 is determined based on the center position P2 of the detector and the tube position P3. The doctor plans a puncture path on the CT-like image to determine the planned path. The center of the CT-like image is taken as the body center position. It should be noted that the body center position corresponds to the patient center P0 in the physical space. On the CT-like image, the planned needle entry position is determined based on the body center position of the CT-like image. According to the planned path, the planned puncture angle is determined. Then, a puncture needle with a suitable length is selected, and the planned needle tail position is determined according to the planned puncture angle and the length of the puncture needle. The coordinate system in which the planned needle entry position and the planned needle tail position are located is converted into the world coordinate system to obtain the planned needle entry position P5 in the world coordinate system and the planned needle tail position P4 in the world coordinate system. The positional relationship between the position P1 of the camera and the center position P2 of the detector is fixed.

[0060] Next, the C-arm is adjusted based on the planned puncture angle, and the position of the detector will also change. According to the adjusted position, the center position of the detector in the world coordinate system is determined. Then, based on the center position of the detector, the camera position in the world coordinate system is determined. The planned needle entry position P5 in the world coordinate system and the planned needle tail position P4 in the world coordinate system are converted in coordinates with the camera position as the origin to obtain the puncture needle entry position and the puncture needle tail position in the camera coordinate system. The puncture needle entry position and the puncture needle tail position are projected and converted to obtain the projected needle entry position and the projected needle tail position on the camera plane.

[0061] Then, the camera is used to take a photograph to obtain a photograph image. The photograph image is detected by the puncture needle recognition model to determine the real-time needle entry position and the real-time needle tail position. In the case where the relative position of the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position overlaps with the projected needle entry position, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, the puncture operation is performed according to the real-time needle entry position and the real-time needle tail position at this time, thereby realizing convenient, fast and high-precision puncture navigation.

[0062] It should be noted that the needle entry position in the embodiments of the present specification is the position of the needle tip of the puncture needle on the surface of the human body, rather than the position where the puncture needle has been inserted into the body.

[0063] The embodiments of the present specification provide a puncture guiding method. Please refer to Figure 2 The puncture guiding method can include the following steps:

[0064] S210, a puncture path is planned based on a three-dimensional medical scan image to determine a planned needle entry position and a planned puncture angle.

[0065] Specifically, first, a three-dimensional medical scan image is acquired by scanning using a medical imaging device. Then, a puncture path is planned on the three-dimensional medical scan image according to the patient's condition and specific clinical needs, and the planned puncture path is marked on the three-dimensional medical scan image. According to the marked planned puncture path, a planned needle entry position and a planned puncture angle are determined. The three-dimensional medical scan image can be a CT-like image scanned by DSA.

[0066] In some embodiments, a doctor can manually plan a puncture path according to the structural features displayed on the three-dimensional medical scan image, combined with clinical experience, to determine the planned needle entry position and the planned puncture angle.

[0067] In other embodiments, a path planning algorithm considers the location of the lesion, the surrounding anatomical structure, etc., and automatically plans a puncture path on the three-dimensional medical scan image using a path planning algorithm to determine the planned needle entry position and the planned puncture angle.

[0068] S220, coordinate conversion and projection conversion are performed based on the planned needle entry position, the planned puncture angle, and the length of the puncture needle to obtain a projected needle entry position and a projected needle tail position on the camera plane corresponding to the planned needle entry position.

[0069] S230, based on the projected needle entry position and the projected needle tail position, the puncture operation is guided.

[0070] Specifically, since the needle entry position and the needle tail position captured by the camera are displayed on the camera plane, coordinate conversion and projection conversion are performed using the determined planned needle entry position, planned puncture angle, and length of the puncture needle to convert the planned needle entry position and the planned needle tail position to the corresponding camera plane of the camera to obtain the projected needle entry position and the projected needle tail position on the camera plane. The needle entry position and the needle tail position captured by the camera and the projected needle tail position are displayed on the same plane for comparison to ensure the accuracy of the puncture. Then, the projected needle entry position and the projected needle tail position are matched with the needle entry position and the needle tail position captured by the camera, and based on the real-time feedback of the matching result, the doctor is guided to adjust the needle entry position and the angle of the puncture needle to correct the puncture deviation and realize the guidance of the puncture operation.

[0071] In the above embodiment, first, the puncture path is planned based on the three-dimensional medical scan image to determine the planned needle entry position and the planned puncture angle. Then, the coordinate conversion and projection conversion are performed based on the planned needle entry position, the planned puncture angle, and the puncture needle length to obtain the projected needle entry position and the projected needle tail position on the camera plane corresponding to the planned needle entry position. Finally, the puncture operation is guided based on the projected needle entry position and the projected needle tail position. Through the above embodiment, the position of the bed does not need to be adjusted, thereby simplifying the operation steps, improving the operation efficiency, and improving the accuracy of the puncture guidance by reducing the error caused by the movement of the equipment. In addition, the above embodiment reduces the number of scans, thereby effectively reducing the radiation dose.

[0072] In some embodiments, referring to Figure 3 The guiding of the puncture operation based on the projected needle entry position and the projected needle tail position can include the following steps:

[0073] S310, target detection is performed on the photographed image to determine the real-time needle entry position and the real-time needle tail position.

[0074] S320, the puncture operation is guided based on the relative position of the real-time needle entry position and the projected needle entry position, and the relative position of the real-time needle tail position and the projected needle tail position.

[0075] Specifically, the target detection is performed on the photographed image by using a visual recognition method, such as using a target recognition model, to analyze the features in the photographed image and automatically identify the puncture needle in the photographed image, thereby determining the real-time needle entry position and the real-time needle tail position. The relative position of the real-time needle entry position and the projected needle entry position, and the relative position of the real-time needle tail position and the projected needle tail position are matched, and the matching result of the real-time feedback is used to guide the doctor to adjust the needle entry position and the angle of the puncture needle to correct the puncture deviation, thereby achieving the guidance of the puncture operation.

[0076] In the above embodiment, the target detection is performed on the photographed image to determine the real-time needle entry position and the real-time needle tail position, and the puncture operation is guided based on the relative position of the real-time needle entry position and the projected needle entry position, and the relative position of the real-time needle tail position and the projected needle tail position, thereby ensuring that the final puncture position overlaps with the planned puncture position, thereby realizing a high-precision puncture process.

[0077] In some embodiments, the guiding of the puncture operation based on the relative position of the real-time needle entry position and the projected needle entry position, and the relative position of the real-time needle tail position and the projected needle tail position can include: in a case where the relative position of the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position overlaps with the projected needle entry position, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, the puncture operation is performed.

[0078] Specifically, before performing the puncture operation, a target recognition model is used to detect the target in the photographed image, analyze the features in the photographed image, and automatically identify the puncture needle in the photographed image, so as to determine the real-time needle entry position and the real-time needle tail position. Subsequently, the real-time needle entry position is compared with the projected needle entry position, and the real-time needle tail position is compared with the projected needle tail position, and by analyzing the relative positions thereof, it is determined whether they overlap, so as to ensure the accuracy of the puncture. In the case that the relative position of the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position and the projected needle entry position overlap, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position and the projected needle tail position overlap, the puncture operation is performed. At this time, the puncture path constituted by the needle tip position and the inclination angle of the puncture needle is highly consistent with the planned puncture path, so as to ensure the accuracy of the puncture path of the puncture operation.

[0079] It should be noted that the projected needle entry position and the projected needle tail position are displayed on the display of the medical imaging device as a reference, and provide the necessary visual reference for the doctor. The real-time needle entry position and the real-time needle tail position are displayed in real time on the display of the medical imaging device, so as to ensure that the doctor can monitor the current state and position of the puncture needle at any time, so as to determine the positional relationship between the real-time needle entry position and the real-time needle tail position and the projected needle entry position and the projected needle tail position, and help the doctor to timely adjust the entry position and the inclination angle of the puncture needle.

[0080] In some embodiments, the real-time needle entry position is compared with the projected needle entry position, and the relative error therebetween is analyzed. Then, it is determined whether the relative error is within a preset acceptable range, so as to ensure the puncture accuracy. When the error between the real-time needle entry position and the projected needle entry position is within the allowable range, it is considered that the relative position of the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position and the projected needle entry position overlap.

[0081] The real-time needle tail position is compared with the projected needle tail position, and the relative error therebetween is analyzed. Then, it is determined whether the relative error is within a preset acceptable range, so as to ensure the puncture accuracy. When the error between the real-time needle tail position and the projected needle tail position is within the allowable range, it is considered that the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position and the projected needle tail position overlap.

[0082] In the above embodiments, in the case that the relative position of the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position and the projected needle entry position overlap, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position and the projected needle tail position overlap, the puncture operation is performed, so as to ensure that the final puncture position overlaps with the planned puncture position, thereby realizing a high-precision puncture process.

[0083] In some embodiments, please refer to Figure 4The three-dimensional medical scan image includes a body center position, the puncture path planning is performed based on the three-dimensional medical scan image, the planned needle entry position and the planned puncture angle are determined, and the method can include the following steps:

[0084] S410, the puncture path planning is performed on the three-dimensional medical scan image, and the planned path is determined.

[0085] S420, the planned puncture angle is determined according to the planned path.

[0086] S430, the planned needle entry position is determined based on the body center position of the three-dimensional medical scan image.

[0087] Specifically, the puncture path planning is performed on the three-dimensional medical scan image according to the specific position of the lesion and the actual situation, a suitable path is determined, and the planned path is obtained. The needle entry position is determined according to the planned path, then a line segment parallel to the bed is drawn from the needle entry position, and the included angle between the line segment and the planned path is calculated, which is used as the planned puncture angle. The center position of the three-dimensional medical scan image is determined based on the three-dimensional medical scan image scanned by the medical imaging device, so as to obtain the body center position. In the three-dimensional medical scan image, the position of the intersection between the planned path and the patient is determined according to the position relationship between the intersection and the body center position, and the position of the intersection between the planned path and the patient is determined as the planned needle entry position. It should be noted that the body center position can be the patient center.

[0088] In the above embodiment, the puncture path planning is performed on the three-dimensional medical scan image, the planned path is determined, the planned puncture angle is determined according to the planned path, and the planned needle entry position is determined based on the body center position of the three-dimensional medical scan image, which helps to improve the accuracy of puncture and reduce errors in actual operation.

[0089] In some embodiments, referring to Figure 5 The planned needle entry position and the projected needle tail position on the camera plane are obtained by coordinate conversion and projection conversion based on the planned needle entry position, the planned puncture angle and the length of the puncture needle, and the method can include the following steps:

[0090] S510, the planned needle tail position is determined according to the planned puncture angle and the length of the puncture needle.

[0091] S520, the projected needle entry position and the projected needle tail position on the camera plane are obtained by coordinate conversion and projection conversion based on the planned needle entry position and the planned needle tail position.

[0092] Specifically, a puncture needle with a proper length is selected according to the specific condition and clinical requirement of the patient. After the length of the puncture needle required is determined, a scale factor between the image and the actual situation is calculated based on the comparison between the reference object in the three-dimensional medical scan image and the actual length thereof. By using the scale factor, the length of the actual puncture needle is converted into the corresponding puncture needle length in the three-dimensional medical scan image according to the calculated scale factor. Then, in the three-dimensional medical scan image, the corresponding puncture needle length in the three-dimensional medical scan image is taken as the starting point of the planned needle entry position, and a planned puncture angle is inclined, and the planned needle tail position is determined by using the corresponding puncture needle length in the three-dimensional medical scan image and the direction vector corresponding to the planned puncture angle. Since the needle entry position and the needle tail position obtained by the camera shooting are displayed on the camera plane, the planned needle entry position and the planned needle tail position are converted to the camera plane corresponding to the camera by using the determined planned needle entry position and the planned needle tail position for coordinate conversion and projection conversion, so as to obtain the projected needle entry position and the projected needle tail position on the camera plane, so that the needle entry position and the needle tail position obtained by the subsequent camera shooting are compared with the projected needle entry position and the projected needle tail position displayed on the same plane, so as to ensure the accuracy of the puncture.

[0093] In the above embodiment, the planned needle tail position is determined according to the planned puncture angle and the length of the puncture needle, and the coordinate conversion and the projection conversion are performed based on the planned needle entry position and the planned needle tail position to obtain the projected needle entry position and the projected needle tail position on the camera plane, which helps to improve the accuracy of the puncture and reduce errors in actual operation.

[0094] In some embodiments, referring to Figure 6 , the coordinate conversion and the projection conversion based on the planned needle entry position and the planned needle tail position to obtain the projected needle entry position and the projected needle tail position on the camera plane can include the following steps:

[0095] S610, determining the camera position based on the detector center position.

[0096] S620, performing coordinate conversion on the planned needle entry position and the planned needle tail position with the camera position as the origin to obtain the puncture needle entry position and the puncture needle tail position in the camera coordinate system.

[0097] S630, performing projection conversion on the puncture needle entry position and the puncture needle tail position to obtain the projected needle entry position and the projected needle tail position on the camera plane.

[0098] Specifically, after adjusting the C-arm, the detector center position at this time is determined based on the coordinate system built in the medical imaging device. Then, according to the spatial relationship and geometric configuration between the camera and the detector, the camera position is determined by using the detector center position. The camera position is converted from the medical imaging device coordinate system to the world coordinate system to determine the position of the camera in the world coordinate system, thereby obtaining the camera position in the world coordinate system. The planned needle entry position and the planned needle tail position are converted from the image coordinate system to the world coordinate system to obtain the planned needle entry position and the planned needle tail position in the world coordinate system. Then, taking the camera position in the world coordinate system as the origin, the planned needle entry position and the planned needle tail position in the world coordinate system are coordinate-converted to obtain the puncture needle entry position and the puncture needle tail position in the camera coordinate system. Then, using the intrinsic matrix of the camera, the puncture needle entry position and the puncture needle tail position are projected and converted to obtain the projected needle entry position and the projected needle tail position on the camera plane, so as to draw the planned puncture needle on the camera plane. The projected needle entry position and the projected needle tail position serve as the reference for subsequent real-time matching, and ensure the accuracy of the needle entry point and the needle tail point in the actual puncture operation. The camera coordinate system can be understood as the coordinate system of the camera field of view.

[0099] In some embodiments, the medical imaging device is equipped with a camera when it is shipped. In this case, the spatial relationship and geometric configuration between the detector center and the camera are set when the medical imaging device is shipped and are fixed in the medical imaging device.

[0100] In other embodiments, the medical imaging device is not equipped with a camera. In this case, the camera needs to be installed on the medical imaging device subsequently, and the spatial relationship and geometric configuration between the detector center and the camera are determined through a calibration process.

[0101] In the above embodiments, the camera position is determined based on the detector center position, the puncture needle entry position and the puncture needle tail position in the camera coordinate system are obtained by coordinate-converting the planned needle entry position and the planned needle tail position with the camera position as the origin, and the projected needle entry position and the projected needle tail position on the camera plane are obtained by projecting and converting the puncture needle entry position and the puncture needle tail position, so that the actual puncture position can be intuitively compared in actual operation.

[0102] In some embodiments, the detector is located at one end of the C-arm, and determining the detector center position can include: adjusting the C-arm based on the planned puncture angle to determine the detector center position.

[0103] Specifically, the C-arm is adjusted according to the planned puncture angle, so that the angle between the center point of the detector at one end of the C-arm and the tube is consistent with the planned puncture angle, and the line connecting the center point of the detector and the tube is parallel to the planned path. In some embodiments, after the puncture path for performing the puncture is determined, further verification needs to be performed using a medical imaging device. The angle of the detector determined by adjusting the C-arm based on the planned puncture angle can accurately align the preset puncture region. By scanning again through the medical imaging device, it is determined whether the puncture path can act on the lesion, so as to determine whether the puncture path for performing the puncture is accurate.

[0104] After adjusting the C-arm based on the planned puncture angle, the position of the center of the detector in the coordinate system of the medical imaging device is determined based on the coordinate system built in the medical imaging device, to obtain the center position of the detector. For example, the coordinate system of the medical imaging device usually takes the geometric center of the device as the reference point.

[0105] In the above embodiments, the C-arm is adjusted based on the planned puncture angle, and the center position of the detector is determined, so as to subsequently convert the camera, the planned needle entry position, and the planned needle tail position into the same coordinate system to determine the relative positions.

[0106] In some embodiments, referring to Figure 7 Target detection is performed on the photographed image to determine the real-time needle entry position and the real-time needle tail position, which can include the following steps:

[0107] S710, a photographed image is obtained.

[0108] S720, a puncture needle recognition model is used to detect the puncture needle in the photographed image to determine the real-time needle entry position and the real-time needle tail position.

[0109] The photographed image is obtained by a camera installed on the medical imaging device.

[0110] Specifically, first, a photographed image is obtained by photographing through a camera installed on the medical imaging device. Then, a puncture needle recognition model is used to detect the puncture needle in the photographed image, analyze the features in the photographed image, and automatically recognize the puncture needle in the photographed image, so as to determine the real-time needle entry position and the real-time needle tail position. The puncture needle recognition model can be a target detection model determined based on a convolutional neural network (CNN) or other deep learning technology, such as a YOLO (You Only Look Once) network or a Faster R-CNN network.

[0111] Exemplarily, in order to construct a high-quality sample set, a large number of puncture needle photos should be collected. The sample set should cover different angles (such as front view, side view, and top view) to comprehensively show the appearance of the puncture needle. The sample set can also include different positions of the puncture needle, such as the needle head, the needle tail, and cover various lengths of the puncture needle, such as short needles, medium-length needles, and long needles. The shooting should be carried out under various background information, such as different types of medical imaging devices (such as ultrasound, CT, and MRI devices) to simulate different scenarios in actual use. In addition, various lighting conditions should be recorded, such as natural light, different intensity of artificial light, and shadow effect, to ensure that the performance of the puncture needle under various lighting environments is captured. Through such a diversified sample set, the characteristics of the puncture needle can be deeply researched, and more comprehensive feature extraction and more accurate model training can be achieved. The sample set is labeled with the puncture needle, so as to determine the label of the sample set. For example, the puncture needle can be labeled in a manual manner.

[0112] The sample set is input into the initial puncture needle recognition model to extract the features of the puncture needle, and the training puncture needle can be obtained. Then, based on the label corresponding to the sample set and the training puncture needle, the loss value of the initial puncture needle recognition model can be determined, and the parameters of the initial puncture needle recognition model are updated based on the model loss value. By analogy, the updated initial puncture needle recognition model is continuously trained, and when the model training stopping condition is reached, the trained puncture needle recognition model can be obtained. The model training stopping condition can be that the model loss value converges, or the training round reaches a preset number of rounds.

[0113] It should be noted that, referring to Figure 1a , the camera 102 can be installed at one end of the C-arm of the DSA device. Based on the planned puncture angle, the C-arm is adjusted, and the angle between the camera and the puncture area is properly maintained, which helps to reduce the image distortion that may be caused by the angle deviation, thereby reducing the diagnostic error. And the camera at this time can cover the approximate area of the patient that needs to be punctured, so as to obtain the shooting image and the reference image for detecting the puncture needle.

[0114] In the above embodiments, the shooting image is obtained, the puncture needle detection is performed on the shooting image by using the puncture needle recognition model, the real-time needle entry position and the real-time needle tail position are determined, and the basis for determining the position of the puncture is provided.

[0115] In some embodiments, referring to Figure 8 , in the case that the relative position between the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position overlaps with the projected needle entry position, and the relative position between the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, the puncture operation can include the following steps:

[0116] S810, in the case that the relative position of the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position and the projected needle entry position overlap, obtaining a reference image.

[0117] The reference image is obtained by the camera under the condition that the real-time needle entry position is kept unchanged.

[0118] Specifically, before the puncture is performed, the target recognition model is used to perform target detection on the photographed image, analyze the features in the photographed image, and automatically identify the puncture needle in the photographed image, so as to determine the real-time needle entry position. Then, the real-time needle entry position is compared with the projected needle entry position, and by analyzing the relative position of the two, it is determined whether they overlap, so as to ensure the accuracy of the puncture. In the case that the relative position of the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position and the projected needle entry position overlap, in order to further verify and ensure the accuracy of the puncture, a reference image needs to be obtained by the camera under the condition that the real-time needle entry position is kept unchanged. Illustratively, the puncture needle recognition model is used to perform puncture needle detection on the photographed image, analyze the features in the photographed image, and automatically identify the puncture needle in the photographed image, so as to determine the real-time needle entry position.

[0119] In some embodiments, the real-time needle entry position is compared with the projected needle entry position, and the relative error between the two is analyzed. Then, it is determined whether the relative error is within a preset acceptable range, so as to ensure the puncture accuracy. When the error between the real-time needle entry position and the projected needle entry position is within the allowed range, it is considered that the relative position of the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position and the projected needle entry position overlap.

[0120] S820, performing target detection on the reference image to determine a real-time needle tail position.

[0121] S830, in the case that the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position and the projected needle tail position overlap, performing the puncture operation.

[0122] Specifically, after obtaining the reference image, the target recognition model is used to detect the target in the reference image, analyze the features in the reference image, and automatically identify the puncture needle in the captured image, so as to determine the real-time needle tail position. Then, the real-time needle tail position is compared with the projected needle tail position, and by analyzing the relative positions of the two, it is determined whether they overlap, so as to ensure the accuracy of the puncture. In the case where the relative positions of the real-time needle tail position and the projected needle tail position represent that the real-time needle tail position and the projected needle tail position overlap, the puncture operation is performed. At this time, the puncture path formed by the needle tip position and the inclination angle of the puncture needle is highly consistent with the planned puncture path, thereby ensuring the accuracy of the puncture path of the puncture operation. Illustratively, the puncture needle recognition model is used to detect the puncture needle in the reference image, analyze the features in the reference image, and automatically identify the puncture needle in the reference image, so as to determine the real-time needle tail position.

[0123] In the above embodiments, in the case where the relative positions of the real-time needle tail position and the projected needle tail position represent that the real-time needle tail position and the projected needle tail position overlap, the reference image is obtained, the target in the reference image is detected, and the real-time needle tail position is determined. In the case where the relative positions of the real-time needle tail position and the projected needle tail position represent that the real-time needle tail position and the projected needle tail position overlap, the puncture operation is performed, and the final puncture position is ensured to overlap with the planned puncture position, thereby realizing a high-precision puncture process.

[0124] The embodiments of the present specification also provide a puncture guiding method. The three-dimensional medical scan image includes a body center position, and a detector is located at one end of a C-arm. Illustratively, refer to Figure 9 The puncture guiding method can include the following steps:

[0125] S902, a puncture path is planned for the three-dimensional medical scan image, and a planned path is determined.

[0126] S904, a planned puncture angle is determined according to the planned path.

[0127] S906, a planned needle insertion position is determined based on the body center position of the three-dimensional medical scan image.

[0128] S908, a planned needle tail position is determined according to the planned puncture angle and the length of the puncture needle.

[0129] S910, the C-arm is adjusted based on the planned puncture angle to determine a detector center position.

[0130] S912, a camera position is determined based on the detector center position.

[0131] S914, the planned needle insertion position and the planned needle tail position are converted in coordinates with the camera position as the origin, to obtain a puncture needle insertion position and a puncture needle tail position in a camera coordinate system.

[0132] S916, projection conversion is performed on the puncture needle position and the puncture needle tail position to obtain a projected needle entry position and a projected needle tail position on a camera plane.

[0133] S918, a photographed image is obtained.

[0134] The photographed image is obtained by a camera installed on the medical imaging device.

[0135] S920, a puncture needle detection is performed on the photographed image by using a puncture needle recognition model to determine a real-time needle entry position and a real-time needle tail position.

[0136] S922, when a relative position between the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position and the projected needle entry position overlap, a reference image is obtained.

[0137] The reference image is obtained by the camera while keeping the real-time needle entry position unchanged.

[0138] S924, a target detection is performed on the reference image to determine the real-time needle tail position.

[0139] S926, when a relative position between the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position and the projected needle tail position overlap, a puncture operation is performed.

[0140] The present specification provides a puncture guiding device 1000, please refer to Figure 10 The puncture guiding device 1000 comprises a planning data determination module 1010, a projection data determination module 1020, and a puncture operation guiding module 1030.

[0141] The planning data determination module 1010 is configured to determine a planned needle entry position and a planned puncture angle based on a three-dimensional medical scan image.

[0142] The projection data determination module 1020 is configured to perform coordinate conversion and projection conversion based on the planned needle entry position, the planned puncture angle, and a puncture needle length to obtain a projected needle entry position and a projected needle tail position corresponding to the planned needle entry position on a camera plane.

[0143] The puncture operation guiding module 1030 is configured to guide a puncture operation based on the projected needle entry position and the projected needle tail position.

[0144] For specific description of the puncture guiding device, please refer to the description of the puncture guiding method above, which will not be repeated here.

[0145] In some embodiments, a medical imaging device is provided, comprising a memory having a computer program stored therein, and a processor which, when executing the computer program, implements the method steps of the above-described embodiments.

[0146] An embodiment of the present specification provides a computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the steps of the method in any of the above-described embodiments.

[0147] An embodiment of the present specification provides a computer program product comprising instructions which, when executed by a processor of a computer device, enable the computer device to perform the steps of the method of any of the above-described embodiments.

[0148] It should be noted that the logic and / or steps represented in the flowcharts and / or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logic functions, and can be specifically embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or in conjunction with which the instructions can be executed. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical apparatus), a portable computer diskette (magnetic apparatus), a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (optical apparatus), and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable medium upon which the program is printed, as the program can be electronically captured, for example, via an optical scanner, then compiled, interpreted, or otherwise processed in the electronic manner into a useable form, and stored in the computer memory.

Claims

1. A medical imaging apparatus comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor implements the steps of the puncture guiding method when executing the computer program, and the method comprises: planning a puncture path based on a three-dimensional medical scan image, determining a planned needle entry position and a planned puncture angle; performing coordinate conversion and projection conversion based on the planned needle entry position, the planned puncture angle, and a puncture needle length, to obtain a projected needle entry position and a projected needle tail position on a camera plane corresponding to the planned needle entry position, comprising: determining a planned needle tail position according to the planned puncture angle and the puncture needle length, determining a camera position based on a detector center position, performing coordinate conversion on the planned needle entry position and the planned needle tail position with the camera position as the origin to obtain a puncture needle entry position and a puncture needle tail position in a camera coordinate system, and performing projection conversion on the puncture needle entry position and the puncture needle tail position to obtain the projected needle entry position and the projected needle tail position on the camera plane; guiding a puncture operation based on the projected needle entry position and the projected needle tail position.

2. The medical imaging device of claim 1, wherein, The guiding of the puncture operation based on the projected needle entry position and the projected needle tail position comprises: performing target detection on a captured image to determine a real-time needle entry position and a real-time needle tail position; guiding the puncture operation based on the relative positions of the real-time needle entry position and the projected needle entry position, and the relative positions of the real-time needle tail position and the projected needle tail position.

3. The medical imaging device of claim 2, wherein, The guiding of the puncture operation based on the relative positions of the real-time needle entry position and the projected needle entry position, and the relative positions of the real-time needle tail position and the projected needle tail position comprises: performing the puncture operation when the relative positions of the real-time needle entry position and the projected needle entry position indicate that the real-time needle entry position overlaps the projected needle entry position, and the relative positions of the real-time needle tail position and the projected needle tail position indicate that the real-time needle tail position overlaps the projected needle tail position.

4. The medical imaging device of claim 1, wherein, The three-dimensional medical scan image comprises a body center position, and the planning of the puncture path based on the three-dimensional medical scan image to determine the planned needle entry position and the planned puncture angle comprises: planning a puncture path based on the three-dimensional medical scan image to determine a planned path; determining the planned puncture angle according to the planned path; determining the planned needle entry position based on the body center position of the three-dimensional medical scan image.

5. The medical imaging device of claim 1, wherein, The detector is located at one end of a C-arm, and the determination of the detector center position comprises: adjusting the C-arm based on the planned puncture angle to determine the detector center position.

6. The medical imaging device of claim 2, wherein, The target detection on the captured image to determine the real-time needle entry position and the real-time needle tail position comprises: obtaining the captured image, wherein the captured image is obtained by a camera installed on a medical imaging device; performing puncture needle detection on the captured image by using a puncture needle recognition model to determine the real-time needle entry position and the real-time needle tail position.

7. The medical imaging device of claim 3, wherein, In a case where the relative position of the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position overlaps with the projected needle entry position, and the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, the puncture operation is performed, including: In a case where the relative position of the real-time needle entry position and the projected needle entry position indicates that the real-time needle entry position overlaps with the projected needle entry position, a reference image is obtained; wherein the reference image is obtained by a camera under the condition that the real-time needle entry position is kept unchanged; Target detection is performed on the reference image to determine a real-time needle tail position; In a case where the relative position of the real-time needle tail position and the projected needle tail position indicates that the real-time needle tail position overlaps with the projected needle tail position, the puncture operation is performed.

8. A puncture guide device, characterized by, The device includes: A planning data determination module configured to determine a planned needle entry position and a planned puncture angle based on a three-dimensional medical scan image; A projection data determination module configured to perform coordinate conversion and projection conversion based on the planned needle entry position, the planned puncture angle, and a puncture needle length to obtain a projected needle entry position and a projected needle tail position on a camera plane corresponding to the planned needle entry position, including determining a planned needle tail position according to the planned puncture angle and the puncture needle length, determining a camera position based on a detector center position, performing coordinate conversion on the planned needle entry position and the planned needle tail position with the camera position as an origin to obtain a puncture needle entry position and a puncture needle tail position in a camera coordinate system, and performing projection conversion on the puncture needle entry position and the puncture needle tail position to obtain the projected needle entry position and the projected needle tail position on the camera plane; A puncture operation guiding module configured to guide a puncture operation based on the projected needle entry position and the projected needle tail position.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the puncture guiding method, and the method includes: Determining a planned needle entry position and a planned puncture angle based on a three-dimensional medical scan image; Performing coordinate conversion and projection conversion based on the planned needle entry position, the planned puncture angle, and a puncture needle length to obtain a projected needle entry position and a projected needle tail position on a camera plane corresponding to the planned needle entry position, including determining a planned needle tail position according to the planned puncture angle and the puncture needle length, determining a camera position based on a detector center position, performing coordinate conversion on the planned needle entry position and the planned needle tail position with the camera position as an origin to obtain a puncture needle entry position and a puncture needle tail position in a camera coordinate system, and performing projection conversion on the puncture needle entry position and the puncture needle tail position to obtain the projected needle entry position and the projected needle tail position on the camera plane; Guiding a puncture operation based on the projected needle entry position and the projected needle tail position.

Images