Puncture needle position verification method and device, computer equipment and storage medium

Abstract

The invention relates to a puncture needle position verification method and device, computer equipment and a storage medium. According to the method, first position information of a puncture needle positioned by a robot is obtained, a puncture image is obtained, the position of a needle point in the puncture image is recognized according to the first position information, second position information of the puncture needle in the puncture image is obtained, and then the first position information is verified according to the second position information; and obtaining a verification result, and determining target position information according to the verification result. According to the verification method, the needle point positioning of the puncture image is verified through the puncture needle information provided by the robot, the combination of the needle point positioning of an interventional robot hardware system and the needle point positioning of an image algorithm is realized, and the needle point positioning precision and accuracy are greatly improved.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a method, device, computer equipment and storage medium for verifying the position of a puncture needle. Background Art

[0002] CT image-guided percutaneous puncture is a common interventional procedure used for percutaneous biopsy, abscess drainage, and tumor ablation. Due to its low trauma, the demand for minimally invasive puncture is increasing. Among them, the relative position of the tumor and important organs is indeed extremely important for the planning and detection of the puncture path. In order to improve the quality of the puncture operation and increase patient safety, real-time positioning of the puncture needle tip during the operation is a key link.

[0003] Therefore, how to accurately position the puncture needle has become the current real-time needle tip positioning and real-time calculation of puncture direction, which is of great significance to the accuracy of intraoperative puncture and the success rate of the operation. Summary of the invention

[0004] Based on this, it is necessary to provide a puncture needle position verification method, device, computer equipment and storage medium that can effectively verify the position of the puncture needle in the puncture image in order to solve the above technical problems.

[0005] In a first aspect, the present application provides a method for verifying the position of a puncture needle. The method comprises:

[0006] Acquire the puncture image and first position information of the puncture needle positioned by the robot;

[0007] Identify the needle tip position in the puncture image according to the first position information to obtain second position information of the puncture needle in the puncture image;

[0008] The first location information is verified according to the second location information to obtain a verification result, and the target location information is determined according to the verification result.

[0009] In one embodiment, the identifying the needle tip position in the puncture image according to the first position information to obtain the second position information of the puncture needle in the puncture image includes:

[0010] Cropping the puncture image according to the first position information to obtain a cropped image;

[0011] Inputting the cropped image into a segmentation network for segmentation to obtain a segmented image;

[0012] The needle tip position in the segmented image is identified to obtain second position information of the puncture needle in the puncture image.

[0013] In one embodiment, obtaining first position information of the puncture needle positioned by the robot includes:

[0014] Determine the end coordinates of the last section of the robot's mechanical arm according to the length and rotation angle of each section of the robot's mechanical arm;

[0015] Determine the position coordinates of the puncture needle in the robot coordinate system according to the end coordinates of the last section of the robotic arm and the size of the puncture needle;

[0016] According to the position coordinates of the puncture needle in the robot coordinate system and the coordinate system registration information, the first position information of the puncture needle in the image coordinate system is determined; the first position information includes the needle tip position and the needle tail position of the puncture needle.

[0017] In one embodiment, the verifying the first location information according to the second location information to obtain a verification result includes:

[0018] determining a distance between the puncture needle operated by the robot and the puncture needle in the puncture image according to the first position information and the second position information;

[0019] The first location information is verified according to the distance to obtain a verification result.

[0020] In one embodiment, verifying the first location information according to the distance to obtain a verification result includes:

[0021] If the distance is greater than a preset distance threshold, determining that the verification result indicates that the first location information is inaccurate;

[0022] If the distance is not greater than a preset distance threshold, it is determined that the verification result indicates that the first location information is accurate.

[0023] In one embodiment, determining the target location information according to the verification result includes:

[0024] When the verification result indicates that the first location information is accurate, the second location information is determined as the target location information.

[0025] In one of the embodiments, when the distance is greater than a preset distance threshold, the method further includes:

[0026] Determine a reason that causes the first position information to be inaccurate, and determine a next action of the robot according to the reason.

[0027] In one embodiment, determining the next action of the robot according to the cause includes:

[0028] If the reason includes the accumulated error of the robot operation, the robot is instructed to be recalibrated to obtain new coordinate system registration information, so that the robot acquires the next first position information according to the new coordinate system registration information.

[0029] In one embodiment, determining the next action of the robot according to the cause includes:

[0030] If the reason includes deformation of the puncture needle, a needle withdrawal prompt message is displayed to instruct the robot to perform the needle withdrawal operation.

[0031] In a second aspect, the present application also provides a device for verifying the position of a puncture needle. The device comprises:

[0032] An acquisition module, used for acquiring a puncture image and first position information of a puncture needle positioned by the robot;

[0033] an identification module, configured to identify the needle tip position in the puncture image according to the first position information, and obtain second position information of the puncture needle in the puncture image;

[0034] A verification module is used to verify the first location information according to the second location information, obtain a verification result, and determine the target location information according to the verification result.

[0035] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0036] Acquire the puncture image and first position information of the puncture needle positioned by the robot;

[0037] Identify the needle tip position in the puncture image according to the first position information to obtain second position information of the puncture needle in the puncture image;

[0038] The first location information is verified according to the second location information to obtain a verification result, and the target location information is determined according to the verification result.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0040] Acquire the puncture image and first position information of the puncture needle positioned by the robot;

[0041] Identify the needle tip position in the puncture image according to the first position information to obtain second position information of the puncture needle in the puncture image;

[0042] The first location information is verified according to the second location information to obtain a verification result, and the target location information is determined according to the verification result.

[0043] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0044] Acquire the puncture image and first position information of the puncture needle positioned by the robot;

[0045] Identify the needle tip position in the puncture image according to the first position information to obtain second position information of the puncture needle in the puncture image;

[0046] The first location information is verified according to the second location information to obtain a verification result, and the target location information is determined according to the verification result.

[0047] The above-mentioned verification method, device, computer equipment and storage medium for the position of the puncture needle obtain the first position information of the puncture needle located by the robot, obtain the puncture image, and identify the needle tip position in the puncture image according to the first position information to obtain the second position information of the puncture needle in the puncture image, and then verify the first position information according to the second position information to obtain the verification result, and determine the target position information according to the verification result. The above-mentioned verification method verifies the needle tip positioning of the puncture image through the puncture needle information provided by the robot, realizes the combination of the needle tip positioning of the interventional robot hardware system and the needle tip positioning of the image algorithm, and greatly improves the precision and accuracy of the needle tip positioning. In addition, because the robot can provide needle tip positioning in real time, that is, it realizes real-time verification of the needle tip position in the puncture image, it can meet the clinical scenarios with real-time requirements to improve the efficiency of the puncture operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A diagram of an application environment of a method for verifying the position of a puncture needle in one embodiment;

[0049] Figure 2 A schematic diagram of a flow chart of a method for verifying the position of a puncture needle in one embodiment;

[0050] Figure 3 A schematic diagram of a flow chart of a method for verifying the position of a puncture needle in another embodiment;

[0051] Figure 4 A schematic diagram of a flow chart of a method for verifying the position of a puncture needle in another embodiment;

[0052] Figure 5 A schematic diagram of a flow chart of a method for verifying the position of a puncture needle in another embodiment;

[0053] Figure 6 A schematic diagram of a flow chart of a method for verifying the position of a puncture needle in another embodiment;

[0054] Figure 7 A schematic diagram of a flow chart of a method for verifying the position of a puncture needle in another embodiment;

[0055] Figure 8 is a structural block diagram of a device for verifying the position of a puncture needle in one embodiment;

[0056] Fig. 9 is a structural block diagram of a device for verifying the position of a puncture needle in one embodiment;

[0057] Fig.10 is a structural block diagram of a device for verifying the position of a puncture needle in one embodiment;

[0058] Fig.11 is a structural block diagram of a device for verifying the position of a puncture needle in one embodiment;

[0059] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] The method for verifying the position of the puncture needle provided in the embodiment of the present application can be applied to Figure 1 In the verification system shown. The robot 102 is connected to the scanning device 104 by wired or wireless communication. The robot 102 can obtain the puncture image from the scanning device 104, and verify the position of the puncture needle in the puncture image according to the positioning information of the puncture needle provided by the robot 102. The robot 102 can perform puncture surgery, and during the puncture process, the position of the puncture needle can be located in real time, and the real-time positioning information can be obtained. The robot 102 includes a processing device, and the processing device can be but not limited to various personal computers, laptops, smart phones, tablet computers, servers, etc. The scanning device 104 can be various image scanning devices, such as electronic computer tomography (Computed Tomography, CT), magnetic resonance imaging (Magnetic Resonance Imaging, MRI), etc.

[0062] Those skilled in the art will understand that Figure 1 The verification system shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the verification system to which the scheme of the present application is applied. The specific verification system may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0063] In one embodiment, Figure 2 As shown, a method for verifying the position of a puncture needle is provided, and the method is applied to Figure 1 The robot in the example is used to illustrate the following steps:

[0064] S201, obtaining first position information of the puncture needle positioned by the robot, and obtaining a puncture image.

[0065] The first position information is the position information of the puncture needle calculated by the robot according to its own mechanical parameters. The first position information may include the needle tip position and the needle tail position of the puncture needle. The puncture image is an image obtained by scanning the puncture needle during the puncture process.

[0066] In the embodiment of the present application, the robot can calculate the needle tip position and / or needle tail position of the puncture needle in real time during the puncture process using the puncture needle, that is, calculate and obtain the first position information. The robot can also be connected to a scanning device, and the scanning device performs image scanning on the puncture needle during the puncture process of the robot to obtain a puncture image including the puncture needle. After the scanning device scans and obtains each frame of the puncture image, each frame of the puncture image can be transmitted to the processing device in the robot in real time, so that the robot can then verify the position of the puncture needle in the puncture image. Optionally, when the above-mentioned first position information is the position information in the robot coordinate system, after calculating the first position information, the robot can also convert the first position information in the robot coordinate system into the first position information in the image coordinate system according to the coordinate system registration information registered by itself, so as to feed back the first position information to the puncture image for post-processing and other operations.

[0067] S202: Identify the needle tip position in the puncture image according to the first position information to obtain second position information of the puncture needle in the puncture image.

[0068] The second position information includes the needle tip information and needle tail information of the puncture needle in the puncture image.

[0069] In the embodiment of the present application, when the robot obtains the first position information and the puncture image based on the above steps, the position of the needle tip of the puncture needle in the puncture image can be identified by referring to the needle tip information and needle tail information in the first position information, and the second position information including the needle tip position information can be obtained. The identification method may include: first, inputting the first position information and the puncture image together into the needle tip recognition network for needle tip recognition, and directly obtaining the second position information. The needle tip recognition network can be pre-trained for the robot based on the sample set of the puncture image and the corresponding annotated image set, wherein the annotated image set can be obtained by annotating the puncture image corresponding to the needle tip position of the puncture needle provided by the robot. Second, the puncture image is processed according to the first position information to obtain a puncture image that is easy to identify, and then the processed puncture image is input into the needle tip recognition network for needle tip recognition to obtain the second position information.

[0070] S203, verifying the first location information according to the second location information to obtain a verification result, and determining the target location information according to the verification result.

[0071] The verification result includes a verification result indicating that the first location information is accurate and a verification result indicating that the first location information is inaccurate.

[0072] In the embodiment of the present application, when the robot obtains the first position information of the puncture needle calculated by itself and the second position information of the puncture needle in the puncture image, the second position information can be used as reference information to assist in verifying the accuracy of the first position information. Optionally, the robot can verify by checking the consistency of the first position information and the second position information to obtain a verification result; optionally, the robot can also calculate or process the first position information and the second position information to obtain the difference information between the first position information and the second position information, and then verify according to the difference information to obtain a verification result. When the robot obtains the verification result, and the verification result indicates that the first position information is accurate, it means that the second position information previously obtained based on the first position information is also accurate. At this time, the second position information can be determined as the target position information, so that the target position information can be used as the basis for providing guidance information for the later positioning of the puncture needle.

[0073] In the above-mentioned verification method of the puncture needle position, the first position information of the puncture needle located by the robot is obtained, and the puncture image is obtained, and the needle tip position in the puncture image is identified according to the first position information, so as to obtain the second position information of the puncture needle in the puncture image, and then the first position information is verified according to the second position information to obtain the verification result, and the target position information is determined according to the verification result. The above-mentioned verification method verifies the needle tip positioning of the puncture image through the puncture needle information provided by the robot, realizes the combination of the needle tip positioning of the interventional robot hardware system and the needle tip positioning of the image algorithm, and greatly improves the precision and accuracy of the needle tip positioning. In addition, since the robot can provide needle tip positioning in real time, that is, realizes real-time verification of the needle tip position in the puncture image, it can meet the clinical scenarios with real-time requirements, so as to improve the efficiency of the puncture operation.

[0074] In one embodiment, a method for locating a puncture needle in a puncture image is also provided. Figure 3 As shown, the method includes:

[0075] S301, cropping the puncture image according to the first position information to obtain a cropped image.

[0076] In an embodiment of the present application, when the robot calculates the first position information, if the first position information is the position information in the robot coordinate system, the first position information in the robot coordinate system can be further converted into the first position information in the image coordinate system according to the coordinate system registration information, and the size of the puncture needle is determined according to the needle tip information and the needle tail information in the first position information, and then the puncture image is cropped according to the size of the puncture needle to obtain a cropped image, so that the cropped image includes a complete puncture needle and rich information about the puncture needle.

[0077] S302: Input the cropped image into a segmentation network for segmentation to obtain a segmented image.

[0078] The segmentation network may be a pre-trained neural network model, which is used to segment the target tissue in the input image to obtain a corresponding mask image.

[0079] In the embodiment of the present application, the robot can input the cropped image into a pre-trained segmentation network for image segmentation to segment the structure of the puncture needle and obtain a mask image, i.e., a segmented image. It should be noted that the segmentation network can be a lightweight neural network model or a compression model, so that the segmentation network can complete image segmentation in a very short time, achieving the purpose of real-time segmentation, thereby improving the real-time performance of later verification.

[0080] S303, identifying the needle tip position in the segmented image to obtain second position information of the puncture needle in the puncture image.

[0081] In the embodiment of the present application, when the robot obtains the segmented image, it can first restore the size of the segmented image to the size of the puncture image, and identify the needle tip position in the restored segmented image to obtain the second position information of the puncture needle in the puncture image. Optionally, the robot can use a pre-trained positioning network or segmentation network to obtain the needle tip position information of the puncture needle, that is, the second position information of the puncture needle; optionally, the robot can also use a high-dimensional data dimensionality reduction algorithm (Principal Component Analysis, PCA) to identify the needle tip direction in the restored segmented image, thereby obtaining the main puncture direction of the puncture needle in the segmented image to assist in locating the position of the puncture needle.

[0082] The positioning method described in the embodiment of the present application first crops the puncture image based on the first position information of the puncture needle provided by the robot, and then performs image segmentation based on the cropped image, which greatly reduces the amount of calculation required for image segmentation and improves calculation efficiency, thereby improving the real-time performance of puncture needle positioning.

[0083] In one embodiment, the first position information is obtained by calculation by a robot, so the present application also provides a method for obtaining the first position information, such as Figure 4 As shown, the method for obtaining the first location information includes:

[0084] S401, determining the end coordinates of the last section of the robot's robot arm according to the length and rotation angle of each section of the robot's robot arm.

[0085] Among them, the robot can specifically be a multi-degree-of-freedom robotic arm robot, for example, a five-degree-of-freedom robotic arm robot or a six-degree-of-freedom robotic arm robot.

[0086] In an embodiment of the present application, the robot can first calculate the position coordinates of the robotic arm of the first section based on the length and rotation angle of the robotic arm of the first section, and then calculate the position coordinates of the robotic arm of the second section based on the position coordinates of the robotic arm of the first section and combined with the length and rotation angle of the robotic arm of the second section. The position coordinates of the robotic arm of the next section are calculated in this way until the end coordinates of the robotic arm of the last section are obtained.

[0087] S402, determining the position coordinates of the puncture needle in the robot coordinate system according to the end coordinates of the last section of the robot arm and the size of the puncture needle.

[0088] The size of the puncture needle includes the actual needle tip position and needle tail position of the puncture needle. The position coordinates of the puncture needle in the robot coordinate system include the needle tip coordinates and the needle tail coordinates of the puncture needle in the robot coordinate system.

[0089] In the embodiment of the present application, when the robot obtains the end coordinates of the last section of the robotic arm, it can determine the needle tip coordinates of the puncture needle in the robot coordinate system in combination with the actual needle tip position of the puncture needle, and determine the needle tail coordinates of the puncture needle in the robot coordinate system in combination with the actual needle tail position of the puncture needle.

[0090] S403, determining first position information of the puncture needle in the image coordinate system according to the position coordinates of the puncture needle in the robot coordinate system and the coordinate system registration information; the first position information includes the needle tip position and the needle tail position of the puncture needle.

[0091] The coordinate system registration information includes a coordinate system registration matrix, and the coordinate system registration matrix represents the conversion relationship between the robot coordinate system and the image coordinate system.

[0092] In the embodiment of the present application, when the robot calculates the position coordinates of the puncture needle in the robot coordinate system, that is, obtains the needle tip coordinates and needle tail coordinates of the puncture needle in the robot coordinate system, the robot can extract the coordinate system registration matrix from the coordinate system registration information, and based on the coordinate system registration matrix, perform coordinate transformation on the needle tip coordinates of the puncture needle in the robot coordinate system, and perform coordinate transformation on the needle tail coordinates of the puncture needle in the robot coordinate system, specifically transforming them into first position information including needle tip information and needle tail information in the image coordinate system.

[0093] In one embodiment, a method for verifying the position of a puncture needle in a puncture image according to the position information provided by a robot is provided, that is, Figure 5 As shown, the above S203 "verifying the first location information according to the second location information to obtain a verification result" includes:

[0094] S501, determining the distance between the puncture needle located by the robot and the puncture needle in the puncture image according to the first position information and the second position information.

[0095] In an embodiment of the present application, when verifying the first position information, the robot can first extract the needle tip position from the first position information, and extract the needle tip position from the second position information, and then determine the distance between the two needle tip positions, that is, determine the distance between the needle tip position provided by the robot and the needle tip position in the image algorithm, so that the needle tip position provided by the robot can be verified based on the distance afterwards.

[0096] S502: Verify the first location information according to the distance to obtain a verification result.

[0097] In an embodiment of the present application, when the robot obtains the distance between the first position information and the second position information based on the aforementioned steps, it can directly verify the first position information according to the distance value of the distance, or it can evaluate the distance and then verify the first position information according to the evaluation result to obtain a verification result.

[0098] Further, the above S502 "verifies the second location information according to the distance to obtain a verification result", such as Figure 6 As shown, the verification method includes:

[0099] S601, verify the first location information according to the distance to obtain a verification result. If the distance is greater than a preset distance threshold, execute step S602; if the distance is not greater than the preset distance threshold, execute step S603.

[0100] S602: Determine that the verification result indicates that the first location information is inaccurate.

[0101] S603: Determine whether the verification result indicates that the first location information is accurate.

[0102] The preset distance threshold may be determined in advance according to the positioning accuracy of the puncture needle, and is a reference parameter for evaluating whether the first position information is accurate.

[0103] An embodiment of the present application relates to a method for verifying first position information based on a distance value. Specifically, if the distance is greater than a preset distance threshold, it means that the difference between the first position information and the second position information is large, and in this case, the position information of the puncture needle provided by the robot is considered inaccurate; if the distance is not greater than the preset distance threshold, it means that the difference between the first position information and the second position information is small, and in this case, the position information of the puncture needle provided by the robot is considered accurate, and when it is determined that the first position information is accurate, the second position information previously acquired is determined as the target position information.

[0104] In one embodiment, when the verification result indicates that the first location information is inaccurate, the above Figure 6 The method shown also executes the steps of: determining the cause of the inaccurate first position information, and determining the next action of the robot according to the cause.

[0105] Among them, the reasons for the inaccurate first position information include the cumulative error of the robot's positioning system, the deformation of the puncture needle, the mismatch of the puncture needle positioning algorithm, interference from environmental factors, etc.

[0106] In the embodiment of the present application, when it is verified that the first position information is inaccurate, the robot can further analyze the cause of the inaccurate first position information, thereby determining whether it is necessary to continue needle puncture, or whether it is necessary to re-correct the image positioning algorithm, or whether it is necessary to correct the parameters of the robot, etc., according to the cause, that is, determine the next operation of the robot. For example, when the robot determines that the first position information is inaccurate, the robot can self-check its own mechanical parameters, or self-check its own positioning system, to determine whether the accumulated error of the robot's positioning system causes the robot's positioning problem, thereby affecting the accuracy of the first position information provided by the robot. Optionally, the robot can also detect the size of the actual puncture needle to determine whether the puncture needle is deformed. If deformation occurs, it is determined that the deformation of the puncture needle causes the first position information provided by the robot to be inaccurate. Optionally, the robot excludes the inaccuracy of the first position information caused by the accumulated error of the robot's positioning system and the deformation of the puncture needle, and can determine that the inaccuracy of the first position information is caused by other reasons.

[0107] In one embodiment, when the robot analyzes that the cause of the inaccurate first position information includes the accumulated error of the robot's positioning system, the robot can execute the following steps: when the cause includes the accumulated error of the robot's positioning system, the robot is recalibrated to obtain new coordinate system alignment information, so that the robot obtains the next first position information according to the new coordinate system alignment information.

[0108] In the embodiment of the present application, the reasons for the inaccurate first position information include the accumulated error of the robot's positioning system, indicating that after long-term use, the robot's mechanical state has changed, that is, the hardware may be damaged, resulting in the inaccuracy of its own positioning system. Therefore, in this case, the robot can be recalibrated based on the current mechanical state, thereby obtaining new coordinate system registration information for registration, so that when the robot calculates the position information of the next puncture needle in real time, it can calculate based on the new coordinate system registration information, and then obtain accurate puncture needle positioning information. It should be noted that the recalibration process includes: reacquiring the first coordinate of the puncture needle in the new robot coordinate system, and reacquiring the second coordinate of the puncture needle in the image coordinate system, constructing a new coordinate system registration matrix based on the conversion relationship between the first coordinate and the second coordinate, and then obtaining new coordinate system registration information by using the newly constructed coordinate system registration matrix.

[0109] In one embodiment, when the robot analyzes and finds that the cause of the inaccurate first position information includes deformation of the puncture needle, the robot may execute the following steps: when the cause includes deformation of the puncture needle, the robot performs a needle retraction operation.

[0110] In the embodiment of the present application, the reasons for the inaccurate first position information include deformation of the puncture needle, which means that the puncture needle has been deformed after long-term use, that is, it may be damaged, resulting in deviation in the positioning of the puncture needle in the image. Therefore, in this case, the needle puncture cannot be continued, and the robot performs a needle withdrawal operation to avoid affecting the puncture. The robot can also display an alarm message to inform the user that there is a puncture risk and stop the puncture operation. The robot can also replace the puncture needle after the needle withdrawal operation so that the puncture operation can be performed again with a new puncture needle.

[0111] Combining all the above embodiments, Figure 7 As shown, a method for verifying the position of a puncture needle is also provided, the method comprising:

[0112] S701, determining the end coordinates of the last section of the robot's robot arm according to the length and rotation angle of each section of the robot's robot arm.

[0113] S702, determining the coordinates of the puncture needle in the robot coordinate system according to the end coordinates of the last section of the robot arm and the size of the puncture needle.

[0114] S703, determining first position information of the puncture needle in the image coordinate system according to the coordinates of the puncture needle in the robot coordinate system and the coordinate system registration information; the first position information includes the needle tip position and the needle tail position of the puncture needle.

[0115] S704, obtaining a puncture image.

[0116] S705: Crop the puncture image according to the first position information to obtain a cropped image.

[0117] S706, input the cropped image into the segmentation network for segmentation to obtain a segmented image.

[0118] S707, adjusting the size of the segmented image to the size of the original puncture image, and obtaining an adjusted segmented image.

[0119] S708: Identify the needle tip position in the adjusted puncture image according to the first position information to obtain second position information of the puncture needle in the puncture image.

[0120] S709: Determine the distance between the puncture needle located by the robot and the puncture needle in the puncture image according to the first position information and the second position information.

[0121] S710, verify the first location information according to the distance to obtain a verification result. If the distance is greater than a preset distance threshold, execute steps S711 to S715; if the distance is not greater than the preset distance threshold, execute step S716.

[0122] S711, determining that the verification result indicates that the first position information is inaccurate, further determining whether the cause of the inaccurate first position information is the accumulated error of the positioning system of the robot, if so, executing step S712, if not, executing step S713;

[0123] S712: The robot is recalibrated to obtain new coordinate system registration information, so that the robot acquires next first position information according to the new coordinate system registration information.

[0124] S713, determining whether the cause of the inaccurate first position information is the deformation of the puncture needle, if so, executing step S714, if not, executing step S715;

[0125] S714, the robot performs the needle withdrawal operation.

[0126] S715, display a prompt message to inform the user that there is a risk of puncture and stop the puncture.

[0127] S716, determining that the verification result indicates that the first position information is accurate, determining the second position information as the target position information, and continuing to perform puncture according to the target position information.

[0128] The methods described in the above steps are all described in the above content. Please refer to the above description for details and will not be repeated here.

[0129] The method described in the embodiment of the present application realizes effective verification of the position of the puncture needle in the puncture image, and the needle tip position based on the image algorithm is combined with the needle tip position calculated by the interventional robot hardware system to accurately verify the needle tip position of the image algorithm. Moreover, in the case of inaccurate positioning of the verification image algorithm, the robot's positioning system can be corrected in time to achieve the effect of timely guiding the robot to perform correct puncture, thereby improving the efficiency and accuracy of the puncture operation.

[0130] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0131] Based on the same inventive concept, the embodiment of the present application also provides a puncture needle position verification device for implementing the puncture needle position verification method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more puncture needle position verification devices provided below can refer to the limitations of the puncture needle position verification method above, and will not be repeated here.

[0132] In one embodiment, Figure 8 As shown, a device for verifying the position of a puncture needle is provided, comprising:

[0133] The acquisition module 10 is used to acquire the first position information of the puncture needle positioned by the robot, and to acquire the puncture image.

[0134] The identification module 11 is used to identify the needle tip position in the puncture image according to the first position information to obtain the second position information of the puncture needle in the puncture image.

[0135] The verification module 12 is used to verify the first location information according to the second location information, obtain a verification result, and determine the target location information according to the verification result.

[0136] In one embodiment, Fig. 9 As shown, the above identification module 11 includes:

[0137] A cropping unit 110, configured to crop the puncture image according to the first position information to obtain a cropped image;

[0138] A segmentation unit 111 is used to input the cropped image into a segmentation network for segmentation to obtain a segmented image;

[0139] The identification unit 112 is used to identify the needle tip position in the segmented image to obtain second position information of the puncture needle in the puncture image.

[0140] In one embodiment, Fig.10 As shown, the acquisition module 10 includes:

[0141] A first determining unit 100 is used to determine the end coordinates of the last section of the robot's mechanical arm according to the length and rotation angle of each section of the robot's mechanical arm;

[0142] The second determining unit 101 is used to determine the position coordinates of the puncture needle in the robot coordinate system according to the end coordinates of the last section of the robot arm and the size of the puncture needle;

[0143] The third determining unit 102 is used to determine the first position information of the puncture needle in the image coordinate system according to the position coordinates of the puncture needle in the robot coordinate system and the coordinate system registration information; the first position information includes the needle tip position and the needle tail position of the puncture needle.

[0144] In one embodiment, Fig.11 As shown, the verification module 12 includes:

[0145] a fourth determining unit 120, configured to determine a distance between the puncture needle located by the robot and the puncture needle in the puncture image according to the first position information and the second position information;

[0146] The verification unit 121 is used to verify the first location information according to the distance to obtain a verification result.

[0147] In one embodiment, the verification unit 121 is specifically used to determine that the verification result indicates that the first location information is inaccurate when the distance is greater than a preset distance threshold, and to determine that the verification result indicates that the first location information is accurate when the distance is not greater than the preset distance threshold.

[0148] In one embodiment, the verification module 12 is further specifically configured to determine the second location information as the target location information when the verification result indicates that the first location information is accurate.

[0149] In one embodiment, the verification unit 121 is further specifically configured to determine a reason causing the first position information to be inaccurate, and determine a next action of the robot according to the reason.

[0150] In one embodiment, the verification unit 121 is also specifically used to recalibrate the robot to obtain new coordinate system alignment information when the cause includes the accumulated error of the robot's positioning system, so that the robot can obtain the next first position information according to the new coordinate system alignment information.

[0151] In one embodiment, the verification unit 121 is further specifically configured to enable the robot to perform a needle withdrawal operation when the cause includes deformation of the puncture needle.

[0152] Each module in the above-mentioned puncture needle position verification device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0153] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for verifying the position of a puncture needle is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0154] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0155] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0156] Acquiring first position information of the puncture needle positioned by the robot, and acquiring a puncture image;

[0157] Identify the needle tip position in the puncture image according to the first position information to obtain second position information of the puncture needle in the puncture image;

[0158] The first location information is verified according to the second location information to obtain a verification result, and the target location information is determined according to the verification result.

[0159] The computer device provided in the above embodiment has an implementation principle and technical effects similar to those of the above method embodiment, which will not be described in detail here.

[0160] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0161] Acquiring first position information of the puncture needle positioned by the robot, and acquiring a puncture image;

[0162] Identify the needle tip position in the puncture image according to the first position information to obtain second position information of the puncture needle in the puncture image;

[0163] The first location information is verified according to the second location information to obtain a verification result, and the target location information is determined according to the verification result.

[0164] The above embodiment provides a computer-readable storage medium, whose implementation principle and technical effect are similar to those of the above method embodiment, and will not be repeated here.

[0165] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0166] Acquiring first position information of the puncture needle positioned by the robot, and acquiring a puncture image;

[0167] Identify the needle tip position in the puncture image according to the first position information to obtain second position information of the puncture needle in the puncture image;

[0168] The first location information is verified according to the second location information to obtain a verification result, and the target location information is determined according to the verification result.

[0169] The above embodiment provides a computer program product, whose implementation principle and technical effect are similar to those of the above method embodiment, and will not be repeated here.

[0170] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0171] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0172] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for verifying the position of a puncture needle, It is characterized in that The method comprises: Acquiring first position information of the puncture needle positioned by the robot, and acquiring a puncture image; Identify the needle tip position in the puncture image according to the first position information to obtain second position information of the puncture needle in the puncture image; The first location information is verified according to the second location information to obtain a verification result, and the target location information is determined according to the verification result.

2. The method according to claim 1, It is characterized in that The step of identifying the needle tip position in the puncture image according to the first position information to obtain second position information of the puncture needle in the puncture image includes: Cropping the puncture image according to the first position information to obtain a cropped image; Inputting the cropped image into a segmentation network for segmentation to obtain a segmented image; The needle tip position in the segmented image is identified to obtain second position information of the puncture needle in the puncture image.

3. The method according to claim 1, It is characterized in that Acquiring first position information of the puncture needle positioned by the robot includes: Determine the end coordinates of the last section of the robot's mechanical arm according to the length and rotation angle of each section of the robot's mechanical arm; Determine the position coordinates of the puncture needle in the robot coordinate system according to the end coordinates of the last section of the robotic arm and the size of the puncture needle; According to the position coordinates of the puncture needle in the robot coordinate system and the coordinate system registration information, the first position information of the puncture needle in the image coordinate system is determined; the first position information includes the needle tip position and the needle tail position of the puncture needle.

4. The method according to any one of claims 1 to 3, It is characterized in that The verifying the first location information according to the second location information to obtain a verification result includes: determining, according to the first position information and the second position information, a distance between the puncture needle located by the robot and the puncture needle in the puncture image; The first location information is verified according to the distance to obtain a verification result.

5. The method according to claim 4, It is characterized in that The verifying the first location information according to the distance to obtain a verification result includes: If the distance is greater than a preset distance threshold, determining that the verification result indicates that the first location information is inaccurate; If the distance is not greater than a preset distance threshold, it is determined that the verification result indicates that the first location information is accurate.

6. The method according to claim 5, It is characterized in that The determining the target location information according to the verification result includes: When the verification result indicates that the first location information is accurate, the second location information is determined as the target location information.

7. The method according to claim 5 or 6, It is characterized in that When the distance is greater than a preset distance threshold, the method further includes: Determine a reason that causes the first position information to be inaccurate, and determine a next action of the robot according to the reason.

8. The method according to claim 7, It is characterized in that Determining the next action of the robot according to the cause includes: If the cause includes the accumulated error of the positioning system of the robot, the robot is recalibrated to obtain new coordinate system registration information, so that the robot acquires the next first position information according to the new coordinate system registration information.

9. The method according to claim 7, It is characterized in that Determining the next action of the robot according to the cause includes: If the cause includes deformation of the puncture needle, the robot performs a needle withdrawal operation.

10. A device for verifying the position of a puncture needle, It is characterized in that The device comprises: An acquisition module, used to acquire first position information of the puncture needle located by the robot, and to acquire a puncture image; an identification module, configured to identify the needle tip position in the puncture image according to the first position information, and obtain second position information of the puncture needle in the puncture image; A verification module is used to verify the first location information according to the second location information, obtain a verification result, and determine the target location information according to the verification result.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images