Methods, apparatus, computer equipment, and storage media for verifying the location of the puncture needle.

Abstract

This application relates to a method, apparatus, computer device, and storage medium for verifying the position of a puncture needle. The method involves acquiring first position information of the puncture needle located by a robot, acquiring a puncture image, identifying the needle tip position in the puncture image based on the first position information to obtain second position information of the puncture needle in the puncture image, verifying the first position information based on the second position information to obtain a verification result, and determining the target position information based on the verification result. This verification method verifies the needle tip positioning in the puncture image using puncture needle information provided by the robot, combining needle tip positioning from the intervention robot hardware system with needle tip positioning from image algorithms, greatly improving the accuracy and precision of needle tip positioning.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method, apparatus, computer device, and storage medium for verifying the position of a puncture needle. Background Technology

[0002] CT-guided percutaneous puncture is a common interventional procedure used for percutaneous biopsy, abscess drainage, and tumor ablation. Due to its minimally invasive nature, the demand for minimally invasive puncture procedures is increasing. The relative position of the tumor and vital organs is crucial for the planning and detection of the puncture path. To improve the quality of the puncture procedure and increase patient safety, real-time positioning of the puncture needle tip during the procedure is a key step.

[0003] Therefore, accurate positioning of the puncture needle is crucial for real-time needle tip localization and real-time calculation of the puncture direction, which is of great significance for the accuracy of intraoperative puncture and the success rate of surgery. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, and storage medium for verifying the position of a puncture needle in a puncture image, which can effectively verify the position of the puncture needle in the aforementioned technical problem.

[0005] Firstly, this application provides a method for verifying the position of a puncture needle. The method includes:

[0006] Acquire puncture images and the first position information of the puncture needle positioned by the robot;

[0007] The needle tip position in the puncture image is identified based on the first position information to obtain the second position information of the puncture needle in the puncture image;

[0008] The first location information is verified based on the second location information to obtain a verification result, and the target location information is determined based on the verification result.

[0009] In one embodiment, the step of identifying the needle tip position in the puncture image based on the first position information to obtain second position information of the puncture needle in the puncture image includes:

[0010] The puncture image is cropped based on the first location information to obtain a cropped image;

[0011] The cropped image is input into a segmentation network for segmentation to obtain a segmented image;

[0012] The needle tip position in the segmented image is identified to obtain the second position information of the puncture needle in the puncture image.

[0013] In one embodiment, obtaining the first position information of the puncture needle located by the robot includes:

[0014] Based on the length and rotation angle of each robotic arm segment in the robot, determine the end coordinates of the last robotic arm segment in the robot;

[0015] Based on the end-effector coordinates of the last section of the robotic arm and the dimensions of the puncture needle, determine the position coordinates of the puncture needle in the robot coordinate system;

[0016] Based on the position coordinates of the puncture needle in the robot coordinate system and the coordinate system registration information, the first position information of the puncture needle in the image coordinate system is determined; the first position information includes the position of the needle tip and the position of the needle tail.

[0017] In one embodiment, the step of verifying the first location information based on the second location information to obtain a verification result includes:

[0018] Based on the first location information and the second location information, the distance between the puncture needle operated by the robot and the puncture needle in the puncture image is determined;

[0019] The first location information is verified based on the distance to obtain the verification result.

[0020] In one embodiment, verifying the first location information based on the distance to obtain a verification result includes:

[0021] If the distance is greater than a preset distance threshold, then the verification result indicates that the first location information is inaccurate;

[0022] If the distance is not greater than a preset distance threshold, then the verification result indicates that the first location information is accurate.

[0023] In one embodiment, determining the target location information based on the verification result includes:

[0024] If the verification result indicates that the first location information is accurate, the second location information is determined as the target location information.

[0025] In one embodiment, when the distance is greater than a preset distance threshold, the method further includes:

[0026] The cause of the inaccuracy of the first location information is determined, and the next action of the robot is determined based on the cause.

[0027] In one embodiment, determining the robot's next action based on the reason includes:

[0028] If the cause includes cumulative errors in robot operation, the robot is instructed to recalibrate to obtain new coordinate system registration information, so that the robot can obtain the next first position information based on the new coordinate system registration information.

[0029] In one embodiment, determining the robot's next action based on the reason includes:

[0030] If the cause includes deformation of the puncture needle, a needle withdrawal prompt message is displayed to instruct the robot to perform a needle withdrawal operation.

[0031] Secondly, this application also provides a device for verifying the position of a puncture needle. The device includes:

[0032] The acquisition module is used to acquire puncture images and the first position information of the puncture needle positioned by the robot;

[0033] The identification module is used to identify the position of the needle tip in the puncture image based on the first position information, and obtain the second position information of the puncture needle in the puncture image;

[0034] The verification module is used to verify the first location information based on the second location information, obtain a verification result, and determine the target location information based on the verification result.

[0035] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0036] Acquire puncture images and the first position information of the puncture needle positioned by the robot;

[0037] The needle tip position in the puncture image is identified based on the first position information to obtain the second position information of the puncture needle in the puncture image;

[0038] The first location information is verified based on the second location information to obtain a verification result, and the target location information is determined based on the verification result.

[0039] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0040] Acquire puncture images and the first position information of the puncture needle positioned by the robot;

[0041] The needle tip position in the puncture image is identified based on the first position information to obtain the second position information of the puncture needle in the puncture image;

[0042] The first location information is verified based on the second location information to obtain a verification result, and the target location information is determined based on the verification result.

[0043] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0044] Acquire puncture images and the first position information of the puncture needle positioned by the robot;

[0045] The needle tip position in the puncture image is identified based on the first position information to obtain the second position information of the puncture needle in the puncture image;

[0046] The first location information is verified based on the second location information to obtain a verification result, and the target location information is determined based on the verification result.

[0047] The aforementioned method, apparatus, computer equipment, and storage medium for verifying the position of the puncture needle involve acquiring first position information of the puncture needle located by the robot, acquiring a puncture image, identifying the needle tip position in the puncture image based on the first position information to obtain second position information of the puncture needle in the puncture image, verifying the first position information based on the second position information to obtain a verification result, and determining the target position information based on the verification result. This verification method uses the puncture needle information provided by the robot to verify the needle tip positioning in the puncture image, combining needle tip positioning from the interventional robot hardware system with needle tip positioning from image algorithms, greatly improving the accuracy and precision of needle tip positioning. Furthermore, since the robot can provide needle tip positioning in real time, it enables real-time verification of the needle tip position in the puncture image, meeting the real-time requirements of clinical scenarios and improving the efficiency of puncture surgery. Attached Figure Description

[0048] Figure 1 This is a diagram illustrating the application environment of a method for verifying the position of the puncture needle in one embodiment.

[0049] Figure 2 This is a flowchart illustrating a method for verifying the position of the puncture needle in one embodiment;

[0050] Figure 3 This is a flowchart illustrating a method for verifying the position of the puncture needle in another embodiment;

[0051] Figure 4 This is a flowchart illustrating a method for verifying the position of the puncture needle in another embodiment;

[0052] Figure 5 This is a flowchart illustrating a method for verifying the position of the puncture needle in another embodiment;

[0053] Figure 6 This is a flowchart illustrating a method for verifying the position of the puncture needle in another embodiment;

[0054] Figure 7 This is a flowchart illustrating a method for verifying the position of the puncture needle in another embodiment;

[0055] Figure 8 This is a structural block diagram of a device for verifying the position of the puncture needle in one embodiment;

[0056] Figure 9 This is a structural block diagram of a device for verifying the position of the puncture needle in one embodiment;

[0057] Figure 10 This is a structural block diagram of a device for verifying the position of the puncture needle in one embodiment;

[0058] Figure 11 This is a structural block diagram of a device for verifying the position of the puncture needle in one embodiment;

[0059] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] The method for verifying the position of the puncture needle provided in this application embodiment can be applied to, for example... Figure 1 The verification system shown includes a robot 102 connected to a scanning device 104 via wired or wireless communication. The robot 102 can acquire puncture images from the scanning device 104 and verify the position of the puncture needle in the images based on the needle positioning information provided by the robot 102. The robot 102 can perform puncture surgery and can locate the needle in real time during the procedure, obtaining real-time positioning information. The robot 102 includes a processing unit, which can be, but is not limited to, various personal computers, laptops, smartphones, tablets, servers, etc. The scanning device 104 can be various imaging scanning devices, such as computed tomography (CT) and magnetic resonance imaging (MRI).

[0062] Those skilled in the art will understand that Figure 1 The verification system shown in the figure is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the verification system to which the present application is applied. A specific verification system may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0063] In one embodiment, such as Figure 2 As shown, a method for verifying the position of a puncture needle is provided, which is then applied to... Figure 1 Taking the robot in the example, the following steps are included:

[0064] S201, Obtain the first position information of the puncture needle located by the robot, and obtain the puncture image.

[0065] The first position information is the position information of the puncture needle calculated by the robot based on its own mechanical parameters. The first position information may include the position of the needle tip and the position of the needle tail. The puncture image is an image obtained by scanning the puncture needle during the puncture process.

[0066] In this embodiment, during the puncture process using a puncture needle, the robot can calculate the needle tip position and / or needle tail position in real time, i.e., calculate the first position information. The robot can also be connected to a scanning device, which scans the puncture needle during the puncture process to obtain a puncture image including the puncture needle. After the scanning device obtains each frame of the puncture image, it can transmit each frame of the puncture image to the processing device in the robot in real time, so that the robot can later verify the position of the puncture needle in the puncture image. Optionally, when the first position information is position information in the robot coordinate system, after calculating the first position information, the robot can also convert the first position information in the robot coordinate system to the first position information in the image coordinate system according to its own registered coordinate system registration information, so as to feed the first position information back to the puncture image for post-processing operations.

[0067] S202, the needle tip position in the puncture image is identified based on the first position information to obtain the second position information of the puncture needle in the puncture image.

[0068] The second location information includes the needle tip and needle tail information of the puncture needle in the puncture image.

[0069] In this embodiment, when the robot obtains the first position information and the puncture image based on the aforementioned steps, it can identify the needle tip position of the puncture needle in the puncture image by referring to the needle tip and needle tail information in the first position information, thereby obtaining second position information including the needle tip position information. This identification method may include: First, inputting the first position information and the puncture image together into a needle tip recognition network for needle tip recognition to directly obtain the second position information. This needle tip recognition network can be pre-trained by the robot based on a sample set of puncture images and a corresponding set of labeled images. The set of labeled images can be obtained by using the needle tip position of the puncture needle provided by the robot to correspond to labeled puncture images. Second, processing the puncture image according to the first position information to obtain a puncture image that is easy to identify, and then inputting the processed puncture image into the needle tip recognition network for needle tip recognition to obtain the second position information.

[0070] S203, verify the first location information based on the second location information to obtain a verification result, and determine the target location information based on the verification result.

[0071] The verification results include verification results indicating that the first location information is accurate and verification results indicating that the first location information is inaccurate.

[0072] In this embodiment, when the robot obtains its calculated first position information of the puncture needle and the second position information of the puncture needle in the puncture image, it can use the second position information as reference information to assist in verifying the accuracy of the first position information. Optionally, the robot can verify the accuracy by checking the consistency between the first and second position information. Alternatively, the robot can perform calculations or processing on the first and second position information to obtain the difference information between them, and then verify the accuracy based on this difference information. When the robot obtains the verification result, and the verification result indicates that the first position information is accurate, it means that the second position information previously obtained based on the first position information is also accurate. At this point, the second position information can be determined as the target position information so that it can provide guidance for subsequent puncture needle positioning based on the target position information.

[0073] In the aforementioned method for verifying the position of the puncture needle, the first position information of the puncture needle located by the robot and the puncture image are acquired. Based on the first position information, the needle tip position in the puncture image is identified to obtain the second position information of the puncture needle in the puncture image. The first position information is then verified based on the second position information to obtain the verification result. The target position information is then determined based on the verification result. This verification method uses the puncture needle information provided by the robot to verify the needle tip positioning in the puncture image, combining the needle tip positioning of the interventional robot hardware system with the needle tip positioning of the image algorithm, greatly improving the accuracy and precision of needle tip positioning. Furthermore, since the robot can provide needle tip positioning in real time, it enables real-time verification of the needle tip position in the puncture image, meeting the real-time requirements of clinical scenarios and improving the efficiency of puncture surgery.

[0074] In one embodiment, a method for locating the puncture needle in a puncture image is also provided, such as... Figure 3 As shown, the method includes:

[0075] S301, the puncture image is cropped according to the first position information to obtain the cropped image.

[0076] In this embodiment of the application, when the robot calculates the first position information, if the first position information is the position information in the robot coordinate system, the first position information in the robot coordinate system can be further converted into the first position information in the image coordinate system according to the coordinate system registration information. The size of the puncture needle is determined according to the needle tip information and needle tail information in the first position information. Then, the puncture image is cropped according to the size of the puncture needle to obtain the cropped image, so that the cropped image includes the complete puncture needle and rich information about the puncture needle.

[0077] S302, the cropped image is input into the segmentation network for segmentation to obtain the segmented image.

[0078] The segmentation network can be a pre-trained neural network model used to segment target tissues in the input image to obtain the corresponding mask image.

[0079] In this embodiment, the robot can input the cropped image into a pre-trained segmentation network for image segmentation to segment the structure of the puncture needle and obtain a mask image, i.e., a segmented image. It should be noted that the segmentation network can be a lightweight neural network model or a compressed model, enabling it to complete image segmentation in a very short time, achieving real-time segmentation and thus improving the real-time performance of subsequent verification.

[0080] S303, identify the needle tip position in the segmented image to obtain the second position information of the puncture needle in the puncture image.

[0081] In this embodiment, when the robot obtains a segmented image, it can first restore the size of the segmented image to the size of the puncture image, and then identify the needle tip position in the restored segmented image to obtain the second position information of the puncture needle in the puncture image. Optionally, the robot can use a pre-trained localization network or segmentation network to obtain the needle tip position information of the puncture needle, i.e., the second position information of the puncture needle; alternatively, the robot can also use a high-dimensional data dimensionality reduction algorithm (Principal Component Analysis, PCA) to identify the needle tip direction in the restored segmented image, thereby obtaining the main puncture direction of the puncture needle in the segmented image to assist in locating the position of the puncture needle.

[0082] The positioning method described in this application first crops the puncture image based on the first position information of the puncture needle provided by the robot, and then performs image segmentation based on the cropped image. This greatly reduces the amount of computation required for image segmentation, improves computational efficiency, and thus improves the real-time performance of puncture needle positioning.

[0083] In one embodiment, the first location information is calculated by the robot; therefore, this application also provides a method for obtaining the first location information, such as... Figure 4 As shown, the method for obtaining the first location information includes:

[0084] S401, Based on the length and rotation angle of each robotic arm segment in the robot, determine the end coordinates of the last robotic arm segment in the robot.

[0085] The robot can specifically be a multi-degree-of-freedom robotic arm robot, such as a five-degree-of-freedom robotic arm robot or a six-degree-of-freedom robotic arm robot.

[0086] In this embodiment, the robot can first calculate the position coordinates of the first robotic arm based on the length and rotation angle of the first robotic arm, and then calculate the position coordinates of the second robotic arm based on the position coordinates of the first robotic arm and the length and rotation angle of the second robotic arm. This process continues to calculate the position coordinates of the next robotic arm until the end coordinates of the last robotic arm are obtained.

[0087] S402, based on the end-effector coordinates of the last section of the robotic arm and the size of the puncture needle, determine the position coordinates of the puncture needle in the robot coordinate system.

[0088] The dimensions of the puncture needle include the position of the needle tip and the position of the needle tail. The position coordinates of the puncture needle in the robot coordinate system include the coordinates of the needle tip and the coordinates of the needle tail in the robot coordinate system.

[0089] In this embodiment of the application, when the robot obtains the end coordinates of the last section of the robotic arm, it can determine the needle tip coordinates of the puncture needle in the robot coordinate system by combining the needle tip position of the actual puncture needle, and determine the needle tail coordinates of the puncture needle in the robot coordinate system by combining the needle tail position of the actual puncture needle.

[0090] S403, based on the position coordinates of the puncture needle in the robot coordinate system and the coordinate system registration information, determine the first position information of the puncture needle in the image coordinate system; the first position information includes the position of the needle tip and the position of the needle tail.

[0091] The coordinate system registration information includes a coordinate system registration matrix, which represents the transformation relationship between the robot coordinate system and the image coordinate system.

[0092] In this embodiment, when the robot calculates the position coordinates of the puncture needle in the robot coordinate system, that is, the needle tip coordinates and needle tail coordinates of the puncture needle in the robot coordinate system, the robot can extract the coordinate system registration matrix from the coordinate system registration information, and perform coordinate transformation on the needle tip coordinates of the puncture needle in the robot coordinate system and the needle tail coordinates of the puncture needle in the robot coordinate system based on the coordinate system registration matrix, specifically converting them into first position information including needle tip information and needle tail information in the image coordinate system.

[0093] In one embodiment, a method is provided to verify the position of the puncture needle in a puncture image based on position information provided by a robot, i.e., as... Figure 5 As shown, the above-mentioned S203 "verifying the first location information based on the second location information to obtain a verification result" includes:

[0094] S501, based on the first position information and the second position information, determine the distance between the puncture needle located by the robot and the puncture needle in the puncture image.

[0095] In this embodiment of the application, when the robot verifies the first position information, it can first extract the needle tip position from the first position information and extract the needle tip position from the second position information, and then determine the distance between the two needle tip positions, that is, determine the distance between the needle tip position provided by the robot and the needle tip position in the image algorithm, so that the needle tip position provided by the robot can be verified according to the distance later.

[0096] S502, verify the first position information based on the distance to obtain the verification result.

[0097] In this embodiment of the application, when the robot obtains the distance between the first position information and the second position information based on the aforementioned steps, it can directly verify the first position information based on the distance value, or it can evaluate the distance and then verify the first position information based on the evaluation result to obtain the verification result.

[0098] Furthermore, the aforementioned S502 "verifies the second location information based on the distance and obtains the verification result," such as... Figure 6 As shown, the verification method includes:

[0099] S601, verify the first location information based on the distance and obtain the verification result. If the distance is greater than the preset distance threshold, proceed to step S602. If the distance is not greater than the preset distance threshold, proceed to step S603.

[0100] S602, the verification result indicates that the first position information is inaccurate.

[0101] S603, confirming that the verification result indicates the accuracy of the first position information.

[0102] The preset distance threshold can be determined in advance based on the positioning accuracy of the puncture needle position and is a reference parameter used to evaluate whether the first position information is accurate.

[0103] This application embodiment relates to a method for verifying first position information based on a distance value. Specifically, if the distance is greater than a preset distance threshold, it indicates that the difference between the first position information and the second position information is large, and the position information of the puncture needle provided by the robot is considered inaccurate. If the distance is not greater than the preset distance threshold, it indicates that the difference between the first position information and the second position information is small, and the position information of the puncture needle provided by the robot is considered accurate. If the first position information is determined to be accurate, the previously obtained second position information is determined as the target position information.

[0104] In one embodiment, when the verification result indicates that the first location information is inaccurate, the above... Figure 6 The method shown also includes the steps of: determining the cause of the inaccuracy of the first position information, and determining the robot's next action based on the cause.

[0105] The reasons for inaccurate first position information include cumulative errors in the robot's positioning system, deformation of the puncture needle, mismatch in the puncture needle positioning algorithm, and interference from environmental factors.

[0106] In this embodiment, when the first position information is verified to be inaccurate, the robot can further analyze the reasons for the inaccuracy and determine whether to continue needle puncture, whether to revise the image positioning algorithm, or whether to correct the robot's parameters, thus determining the robot's next operation. For example, when the robot determines that the first position information is inaccurate, it can check its own mechanical parameters or its own positioning system to determine whether the cumulative error of the robot's positioning system is causing the positioning problem, thereby affecting the accuracy of the first position information provided by the robot. Optionally, the robot can also detect the size of the actual puncture needle to determine whether the puncture needle has deformed. If deformation occurs, it is determined that the deformation of the puncture needle is causing the inaccuracy of the first position information provided by the robot. Optionally, the robot can determine that the inaccuracy of the first position information is caused by other reasons besides the cumulative error of the robot's positioning system and the deformation of the puncture needle.

[0107] In one embodiment, when the robot analyzes and finds that the cause of the inaccurate first position information includes the cumulative error of the robot's positioning system, the robot can perform the following steps: when the cause includes the cumulative error of the robot's positioning system, the robot performs recalibration to obtain new coordinate system registration information, so that the robot can obtain the next first position information based on the new coordinate system registration information.

[0108] In this embodiment, the inaccuracy of the first position information can be caused by the cumulative error of the robot's positioning system. This indicates that after prolonged use, the robot's mechanical state has changed, meaning there may have been hardware damage, leading to inaccurate positioning. Therefore, in this case, the robot can be recalibrated based on its current mechanical state to obtain new coordinate system registration information for registration. This allows the robot to calculate the position information of the next puncture needle in real time based on the new coordinate system registration information, thus obtaining accurate puncture needle positioning information. It should be noted that the recalibration process includes: re-acquiring the first coordinates of the puncture needle in the new robot coordinate system and the second coordinates of the puncture needle in the image coordinate system; constructing a new coordinate system registration matrix based on the transformation relationship between the first and second coordinates; and then obtaining new coordinate system registration information from this newly constructed coordinate system registration matrix.

[0109] In one embodiment, when the robot analyzes and finds that the cause of inaccurate first position information includes deformation of the puncture needle, the robot can perform the step of: if the cause includes deformation of the puncture needle, the robot performs a needle withdrawal operation.

[0110] In this embodiment, the inaccuracy of the first position information can be caused by deformation of the puncture needle. This indicates that the puncture needle has deformed after prolonged use, potentially resulting in damage and causing a deviation in the positioning of the puncture needle in the image. Therefore, in such cases, needle puncture cannot continue, and the robot performs a needle retraction operation to avoid affecting the puncture. The robot can also display alarm information to inform the user of the puncture risk and stop the puncture operation. The robot can also replace the puncture needle after the needle retraction operation so that a new puncture needle can be used for subsequent puncture operations.

[0111] In summary, all the above embodiments, such as Figure 7 As shown, a method for verifying the position of the puncture needle is also provided, the method comprising:

[0112] S701, based on the length and rotation angle of each robotic arm segment in the robot, determine the end coordinates of the last robotic arm segment in the robot.

[0113] S702 determines the coordinates of the puncture needle in the robot coordinate system based on the end coordinates of the last section of the robotic arm and the size of the puncture needle.

[0114] S703, based on the coordinates of the puncture needle in the robot coordinate system and the coordinate system registration information, determine the first position information of the puncture needle in the image coordinate system; the first position information includes the position of the needle tip and the position of the needle tail.

[0115] S704, acquire puncture image.

[0116] S705, the puncture image is cropped according to the first position information to obtain the cropped image.

[0117] S706: Input the cropped image into the segmentation network for segmentation to obtain a segmented image.

[0118] S707, adjust the size of the segmented image to match the size of the original puncture image to obtain the adjusted segmented image.

[0119] S708, based on the first position information, the needle tip position in the adjusted puncture image is identified to obtain the second position information of the puncture needle in the puncture image.

[0120] S709, based on the first position information and the second position information, determine the distance between the puncture needle located by the robot and the puncture needle in the puncture image.

[0121] S710, verify the first position information based on the distance and obtain the verification result. If the distance is greater than the preset distance threshold, proceed to steps S711 to S715. If the distance is not greater than the preset distance threshold, proceed to step S716.

[0122] S711, if the verification result indicates that the first position information is inaccurate, further determine whether the cause of the inaccurate first position information is the cumulative error of the robot's positioning system. If yes, proceed to step S712; otherwise, proceed to step S713.

[0123] S712, the robot is recalibrated to obtain new coordinate system registration information, so that the robot can obtain the next first position information based on the new coordinate system registration information.

[0124] S713, determine whether the cause of the inaccurate first position information is the deformation of the puncture needle. If yes, proceed to step S714; otherwise, proceed to step S715.

[0125] S714, the robot performs the needle removal operation.

[0126] S715 displays a warning message to inform the user of the risk of puncture and to stop the puncture.

[0127] S716, confirm that the verification result indicates that the first position information is accurate, determine the second position information as the target position information, and continue puncture based on the target position information.

[0128] The methods described in each of the above steps have been explained in the foregoing. For details, please refer to the foregoing description. They will not be repeated here.

[0129] The method described in this application effectively verifies the position of the puncture needle in the puncture image. It combines the needle tip position calculated by the image algorithm with the needle tip position calculated by the interventional robot hardware system to accurately verify the needle tip position using the image algorithm. Furthermore, if the image algorithm's positioning is inaccurate, the robot's positioning system can be corrected in a timely manner, thus guiding the robot to perform the correct puncture and improving the efficiency and accuracy of the puncture surgery.

[0130] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0131] Based on the same inventive concept, this application also provides a device for verifying the position of a puncture needle to implement the aforementioned method for verifying the position of a puncture needle. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the device for verifying the position of a puncture needle provided below can be found in the limitations of the method for verifying the position of a puncture needle described above, and will not be repeated here.

[0132] In one embodiment, such as Figure 8 As shown, a device for verifying the position of a puncture needle is provided, comprising:

[0133] The acquisition module 10 is used to acquire the first position information of the puncture needle located by the robot, and to acquire the puncture image.

[0134] The identification module 11 is used to identify the position of the needle tip in the puncture image based on the first position information, and obtain the second position information of the puncture needle in the puncture image.

[0135] The verification module 12 is used to verify the first location information based on the second location information, obtain a verification result, and determine the target location information based on the verification result.

[0136] In one embodiment, such as Figure 9 As shown, the above-mentioned identification module 11 includes:

[0137] The cropping unit 110 is used to crop the puncture image according to the first position information to obtain the cropped image;

[0138] Segmentation unit 111 is used to input the cropped image into a segmentation network for segmentation to obtain a segmented image;

[0139] The identification unit 112 is used to identify the needle tip position in the segmented image to obtain the second position information of the puncture needle in the puncture image.

[0140] In one embodiment, such as Figure 10 As shown, the acquisition module 10 includes:

[0141] The first determining unit 100 is used to determine the end coordinates of the last robotic arm in the robot based on the length and rotation angle of each robotic arm segment in the robot.

[0142] The second determining unit 101 is used to determine the position coordinates of the puncture needle in the robot coordinate system based on the end coordinates of the last section of the robotic arm and the size of the puncture needle.

[0143] The third determining unit 102 is used to determine the first position information of the puncture needle in the image coordinate system based on the position coordinates of the puncture needle in the robot coordinate system and the coordinate system registration information; the first position information includes the needle tip position and the needle tail position of the puncture needle.

[0144] In one embodiment, such as Figure 11 As shown, the verification module 12 includes:

[0145] The fourth determining unit 120 is used to determine the distance between the puncture needle located by the robot and the puncture needle in the puncture image based on the first position information and the second position information.

[0146] The verification unit 121 is used to verify the first location information based on the distance and obtain a verification result.

[0147] In one embodiment, the verification unit 121 is specifically used to determine that the verification result indicates the first location information is inaccurate when the distance is greater than a preset distance threshold, and to determine that the verification result indicates the first location information is accurate when the distance is not greater than the preset distance threshold.

[0148] In one embodiment, the verification module 12 is further configured to determine the second location information as the target location information when the verification result indicates that the first location information is accurate.

[0149] In one embodiment, the verification unit 121 is further configured to determine the cause of the inaccuracy of the first position information and determine the robot's next action based on the cause.

[0150] In one embodiment, the verification unit 121 is further configured to, when the cause includes the cumulative error of the robot's positioning system, recalibrate the robot to obtain new coordinate system registration information, so that the robot can obtain the next first position information based on the new coordinate system registration information.

[0151] In one embodiment, the verification unit 121 is further configured to allow the robot to perform a needle withdrawal operation when the cause includes deformation of the puncture needle.

[0152] Each module in the aforementioned verification device for the puncture needle position can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0153] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for verifying the location of a puncture needle. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0154] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0155] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0156] Obtain the first position information of the puncture needle located by the robot, and obtain the puncture image;

[0157] The needle tip position in the puncture image is identified based on the first position information to obtain the second position information of the puncture needle in the puncture image;

[0158] The first location information is verified based on the second location information to obtain a verification result, and the target location information is determined based on the verification result.

[0159] The computer device provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0160] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0161] Obtain the first position information of the puncture needle located by the robot, and obtain the puncture image;

[0162] The needle tip position in the puncture image is identified based on the first position information to obtain the second position information of the puncture needle in the puncture image;

[0163] The first location information is verified based on the second location information to obtain a verification result, and the target location information is determined based on the verification result.

[0164] The computer-readable storage medium provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0165] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0166] Obtain the first position information of the puncture needle located by the robot, and obtain the puncture image;

[0167] The needle tip position in the puncture image is identified based on the first position information to obtain the second position information of the puncture needle in the puncture image;

[0168] The first location information is verified based on the second location information to obtain a verification result, and the target location information is determined based on the verification result.

[0169] The computer program product provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0170] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0171] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0172] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A device for verifying the position of a puncture needle, characterized in that, The device for verifying the position of the puncture needle includes: The acquisition module is used to acquire the first position information of the puncture needle located by the robot, and to acquire the puncture image; the first position information includes the position of the needle tip and the position of the needle tail. The identification module is used to identify the position of the needle tip in the puncture image based on the first position information, and obtain the second position information of the puncture needle in the puncture image; The verification module determines the distance between the puncture needle located by the robot and the puncture needle in the puncture image based on the first location information and the second location information; verifies the first location information based on the distance to obtain a verification result; if the verification result indicates that the first location information is accurate, the second location information is determined as the target location information; if the verification result indicates that the first location information is inaccurate, the cause of the inaccuracy of the first location information is determined, and the next action of the robot is determined based on the cause.

2. The device for verifying the position of the puncture needle according to claim 1, characterized in that, The identification module includes: The cropping unit is used to crop the puncture image according to the first position information to obtain the cropped image; A segmentation unit is used to input the cropped image into a segmentation network for segmentation to obtain a segmented image; The identification unit is used to identify the needle tip position in the segmented image to obtain the second position information of the puncture needle in the puncture image.

3. The device for verifying the position of the puncture needle according to claim 1, characterized in that, The acquisition module includes: The first determining unit is used to determine the end coordinates of the last robotic arm in the robot based on the length and rotation angle of each robotic arm segment in the robot. The second determining unit is used to determine the position coordinates of the puncture needle in the robot coordinate system based on the end coordinates of the last section of the robotic arm and the size of the puncture needle. The third determining unit is used to determine the first position information of the puncture needle in the image coordinate system based on the position coordinates of the puncture needle in the robot coordinate system and the coordinate system registration information.

4. The device for verifying the position of the puncture needle according to claim 1, characterized in that, The verification module includes: The verification unit is specifically configured to determine that the verification result indicates the first location information is inaccurate when the distance is greater than a preset distance threshold, and to determine that the verification result indicates the first location information is accurate when the distance is not greater than the preset distance threshold.

5. The device for verifying the position of the puncture needle according to claim 4, characterized in that, The verification unit is further specifically used to, when the cause includes the cumulative error of the robot's positioning system, recalibrate the robot to obtain new coordinate system registration information, so that the robot can obtain the next first position information based on the new coordinate system registration information.

6. The device for verifying the position of the puncture needle according to claim 5, characterized in that, The verification unit is also specifically used to enable the robot to perform a needle withdrawal operation when the cause includes deformation of the puncture needle.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the device for verifying the position of the puncture needle as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the device for verifying the position of the puncture needle as described in any one of claims 1 to 6.

Images