Entry point error determination method, system, and surgical robot

Abstract

The application relates to a needle entry point error determination method, system and surgical robot. The method comprises: acquiring a needle channel image obtained by a camera module shooting a target object towards an instrument adapter, and a shooting distance of the camera module shooting the target object towards the needle channel; if the needle channel image contains a target needle entry point, determining a first pixel distance between a center point of the needle channel in the needle channel image and the target needle entry point; according to a preset mapping relationship, determining a first physical distance corresponding to the shooting distance and the first pixel distance, and determining the first physical distance as a needle entry point error of the mechanical arm. The method can improve the accuracy of the needle entry point error determination of the mechanical arm.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a method, system, and surgical robot for determining needle insertion point error. Background Technology

[0002] After registration, neurosurgical robots typically undergo registration verification. During this process, based on 3D medical images of the target object captured by CT (Computed Tomography) equipment, the point that the robotic arm's end effector should reach is determined in the medical image coordinate system corresponding to the 3D medical image, according to the target needle insertion point and target point displayed in the 3D medical image. The coordinate position of this point is then defined as the first coordinate position.

[0003] By registering, the transformation matrix between the medical imaging coordinate system and the robotic arm base coordinate system can be calculated. Based on the transformation matrix, the second coordinate position of the first coordinate position in the robotic arm base coordinate system can be determined. In the robotic arm base coordinate system, the end effector of the robotic arm is controlled to move to the second coordinate position. After the end effector reaches the second coordinate position, a guide can be fixed at the end effector, allowing the puncture needle to move through the channel of the guide.

[0004] If the registration accuracy is very high, the transformation matrix can well reflect the transformation relationship between the medical image coordinate system and the robotic arm base coordinate system. In this case, the point where the puncture needle reaches the surface of the target object should be consistent with the target insertion point. If the registration accuracy is low, the calculated transformation matrix will not be accurate, which will result in a certain distance between the actual point where the puncture needle reaches the surface of the target object and the target insertion point. This distance is the insertion point error.

[0005] In existing technologies, the insertion point error can be measured manually using measuring tools such as rulers. However, this method is easily affected by the operator's subjective factors, leading to inaccurate measurement of the insertion point error. Moreover, long-term use may cause deformation of the puncture needle, further affecting the accuracy of the insertion point error measurement.

[0006] Therefore, current techniques for determining needle insertion point error have limitations in accuracy. Summary of the Invention

[0007] Therefore, it is necessary to provide a method, system, device, computer equipment, computer-readable storage medium, and surgical robot for determining the needle entry point error of a robotic arm that can improve accuracy in addressing the aforementioned technical problems.

[0008] Firstly, this application provides a method for determining the needle insertion point error of a robotic arm. The method includes:

[0009] The camera module captures an image of the target object by pointing towards the needle path of the instrument adapter, and the camera module captures the target object at the shooting distance of the needle path.

[0010] If the needle path image contains the target needle insertion point, then the first pixel distance between the center point of the needle path in the needle path image and the target needle insertion point is determined;

[0011] Based on a preset mapping relationship, a first physical distance corresponding to the shooting distance and the first pixel distance is determined, and the first physical distance is determined as the needle entry point error of the robotic arm.

[0012] In one embodiment, before acquiring the needle path image obtained by the camera module photographing the target object towards the needle path of the instrument adapter, the method further includes:

[0013] Based on the location of the target needle insertion point in the medical image of the target object, and the location of the target point in the medical image of the target object, the location of the verification point is determined in the medical image;

[0014] Based on the registration matrix of the robotic arm and the position of the verification point in the medical image, the position of the verification point in the environment where the robotic arm is located is determined.

[0015] Control the end effector of the robotic arm to move to the location of the verification point in the environment.

[0016] In one embodiment, the method further includes:

[0017] If the needle path image does not contain the target needle insertion point, then determine the second pixel distance between the center point of the needle path in the needle path image and the edge line of the needle path;

[0018] Based on the mapping relationship, a second physical distance corresponding to the shooting distance and the second pixel distance is determined, and it is determined that the needle entry point error is greater than the second physical distance.

[0019] In one embodiment, the method further includes:

[0020] Display the needle path image;

[0021] Receive a first confirmation message or a second confirmation message for the needle path image; the first confirmation message is used to indicate that the needle path image contains the target needle insertion point, and the second confirmation message is used to indicate that the needle path image does not contain the target needle insertion point.

[0022] In one embodiment, upon receiving the second confirmation message, the method further includes:

[0023] Display at least one candidate needle entry point in the needle path image;

[0024] In response to the selection operation for the at least one candidate needle entry point, a target needle entry point in the needle path image is determined.

[0025] In one embodiment, before acquiring the needle path image obtained by the camera module photographing the target object towards the needle path of the instrument adapter, the method further includes:

[0026] The laser module emits laser light from the needle path toward the target object, generating laser spots;

[0027] If the distance between the laser point and the target needle insertion point is less than a preset threshold, then the step of acquiring the needle path image of the target object with the camera module facing the instrument adapter is executed to determine the needle insertion point error of the robotic arm.

[0028] In one embodiment, obtaining the shooting distance at which the camera module shoots the target object towards the needle path includes:

[0029] Obtain the laser ranging distance between the laser module and the laser point;

[0030] Based on the positional relationship between the laser module and the camera module, and the laser ranging distance, the shooting distance at which the camera module shoots the target object towards the needle path is determined.

[0031] Secondly, this application also provides a system for determining the needle entry point error of a robotic arm. The system includes an instrument adapter and a controller; the instrument adapter is equipped with a camera module.

[0032] The camera module is used to take a picture of the target object towards the needle path of the instrument adapter, obtain a needle path image, and send the needle path image to the controller;

[0033] The controller is configured to acquire a needle path image obtained by the camera module shooting the target object towards the needle path of the instrument adapter, and to acquire the shooting distance of the camera module shooting the target object towards the needle path; if the needle path image contains the target needle entry point, then determine the first pixel distance between the center point of the needle path in the needle path image and the target needle entry point; according to a preset mapping relationship, determine a first physical distance corresponding to the shooting distance and the first pixel distance, and determine the first physical distance as the needle entry point error of the robotic arm.

[0034] In one embodiment, the instrument adapter is provided with a laser module; the laser module is used to emit a laser from the needle path toward the target object to generate a laser spot.

[0035] In one embodiment, the controller is further configured to execute a step of acquiring a needle track image of the target object with the camera module facing the instrument adapter based on a trigger command, so as to determine the needle entry point error of the robotic arm; wherein the trigger command is generated when the distance between the laser point and the target needle entry point is less than a preset threshold.

[0036] In one embodiment, the laser module is further configured to determine the laser ranging distance between the laser module and the laser point, and send the laser ranging distance to the controller;

[0037] The controller is further configured to receive the laser ranging distance, and determine the shooting distance at which the camera module takes pictures of the target object towards the needle track based on the positional relationship between the laser module and the camera module, and the laser ranging distance.

[0038] Thirdly, this application also provides a surgical robot. The surgical robot includes a robotic arm and the needle insertion point error determination system for the robotic arm described in the above embodiments.

[0039] Fourthly, this application also provides a device for determining the needle entry point error of a robotic arm. The device includes:

[0040] The acquisition module is used to acquire the needle track image obtained by the camera module taking pictures of the target object with the needle track facing the instrument adapter, and to acquire the shooting distance of the camera module taking pictures of the target object with the needle track facing the needle track;

[0041] A calculation module is used to determine a first pixel distance between the center point of the needle path in the needle path image and the target needle entry point if the needle path image contains the target needle entry point;

[0042] The determination module is used to determine a first physical distance corresponding to the shooting distance and the first pixel distance according to a preset mapping relationship, and to determine the first physical distance as the needle entry point error of the robotic arm.

[0043] Fifthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0044] The camera module captures an image of the target object by pointing towards the needle path of the instrument adapter, and the camera module captures the target object at the shooting distance of the needle path.

[0045] If the needle path image contains the target needle insertion point, then the first pixel distance between the center point of the needle path in the needle path image and the target needle insertion point is determined;

[0046] Based on a preset mapping relationship, a first physical distance corresponding to the shooting distance and the first pixel distance is determined, and the first physical distance is determined as the needle entry point error of the robotic arm.

[0047] Sixthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0048] The camera module captures an image of the target object by pointing towards the needle path of the instrument adapter, and the camera module captures the target object at the shooting distance of the needle path.

[0049] If the needle path image contains the target needle insertion point, then the first pixel distance between the center point of the needle path in the needle path image and the target needle insertion point is determined;

[0050] Based on a preset mapping relationship, a first physical distance corresponding to the shooting distance and the first pixel distance is determined, and the first physical distance is determined as the needle entry point error of the robotic arm.

[0051] The aforementioned method, system, device, computer equipment, storage medium, and surgical robot for determining the needle entry point error of a robotic arm acquires a needle path image of the target object by capturing the needle path of the target object with a camera module facing the instrument adapter, and acquires the shooting distance of the camera module facing the needle path of the target object. If the needle path image contains the target needle entry point, the first pixel distance between the center point of the needle path in the needle path image and the target needle entry point is determined. According to a preset mapping relationship, a first physical distance corresponding to the shooting distance and the first pixel distance is determined, and the first physical distance is determined as the needle entry point error of the robotic arm. When the captured needle path image contains the target needle entry point, the center point of the needle path in the image is taken as the actual needle entry point. The pixel distance between the actual needle entry point and the target needle entry point is calculated, and then the pixel distance is converted into a physical distance according to the mapping relationship between the pixel distance and the physical distance to obtain the actual error distance between the actual needle entry point and the target needle entry point. Since no puncture needle is needed and no manual measurement is required, the accuracy of determining the needle entry point error can be improved. Attached Figure Description

[0052] Figure 1 This is an application environment diagram of a method for determining the needle entry point error of a robotic arm in one embodiment;

[0053] Figure 2 This is a flowchart illustrating a method for determining the needle entry point error of a robotic arm in one embodiment.

[0054] Figure 3 This is a schematic diagram of needle path image acquisition in one embodiment;

[0055] Figure 4 This is a schematic diagram illustrating the determination of needle insertion point error based on needle path images in one embodiment;

[0056] Figure 5 This is a flowchart illustrating the method for determining the needle entry point error of the robotic arm in another embodiment;

[0057] Figure 6 A block diagram of a system for determining the needle entry point error of a robotic arm in one embodiment;

[0058] Figure 7 This is a structural block diagram of the surgical robot in one embodiment;

[0059] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] The method for determining the needle entry point error of the robotic arm provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the instrument adapter 120 is equipped with a camera module 130, which communicates with the controller 140 via a wired or wireless link. The instrument adapter 120 can be, but is not limited to, an adapter for neurosurgical instruments; the camera module 130 can be, but is not limited to, various optical cameras, digital cameras, and optical / digital cameras; and the controller 140 can be a terminal or a server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart vehicle devices; portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices; and the server can be a standalone server or a server cluster consisting of multiple servers.

[0062] In one embodiment, such as Figure 2 As shown, a method for determining the needle entry point error of a robotic arm is provided, which is then applied to... Figure 1 Taking controller 140 as an example, the explanation includes the following steps:

[0063] Step S210: Obtain the needle path image obtained by the camera module shooting the target object towards the needle path of the instrument adapter, and obtain the shooting distance of the camera module shooting the target object towards the needle path.

[0064] The instrument adapter can be installed at the end of the surgical robot's robotic arm. A needle can be fixed on the instrument adapter, and a needle channel can be set on the instrument adapter, in which the needle runs.

[0065] The target object can be the object to be needled, such as the human head.

[0066] The shooting distance can be the shooting distance of the needle track image.

[0067] In practice, a camera module can be installed on the instrument adapter. The camera module can be pointed towards the needle track to capture images of the target object and send the captured images to the controller, allowing the controller to acquire the needle track images. Alternatively, the camera module can be pointed towards the needle track of the instrument adapter to capture images of the target object, and the shooting distance of the needle track images can be measured and sent to the controller, allowing the controller to acquire the shooting distance.

[0068] Step S220: If the needle path image contains the target needle entry point, then determine the first pixel distance between the center point of the needle path in the needle path image and the target needle entry point.

[0069] The target needle insertion point can be the desired needle insertion location on the target object.

[0070] The pixel distance can be the number of pixels between the pixels.

[0071] In practice, it can be detected whether there is a target needle entry point in the captured needle path image. If there is no target needle entry point, it means that the distance between the actual needle entry point and the target needle entry point, i.e. the needle entry point error, is greater than the needle path radius. In this case, the needle entry point error cannot be quantitatively determined. Otherwise, if there is a target needle entry point in the needle path image, the center point of the needle path in the needle path image can be regarded as the actual needle entry point of the needle entry operation to be performed. By calculating the pixel distance between the center point of the needle path and the target needle entry point, the pixel distance between the actual needle entry point and the target needle entry point can be obtained.

[0072] Step S230: Based on the preset mapping relationship, determine the first physical distance corresponding to the shooting distance and the first pixel distance, and determine the first physical distance as the needle entry point error of the robotic arm.

[0073] The first physical distance can be the actual distance between the center point of the needle path and the target needle insertion point.

[0074] In the specific implementation, the mapping relationship d2 = f(d1, d2) can be preset. p ), where d p d1 represents the pixel distance in the needle path image, and d2 represents the actual distance between the camera module and the target needle insertion point. This can be considered as the actual shooting distance at which the camera module faces the needle path of the instrument adapter to capture the target object. p The corresponding actual distance. After obtaining the shooting distance of the needle path image and determining the first pixel distance between the center point of the needle path and the target needle insertion point, the shooting distance can be d1, and the first pixel distance can be d. p According to the mapping relationship d2=f(d1,d p Determine the distance d of the first pixel. p The corresponding first physical distance d2. Since the center point of the needle path can be regarded as the actual needle insertion point of the needle insertion operation to be performed, the first physical distance can be regarded as the actual distance between the actual needle insertion point and the target needle insertion point, that is, the needle insertion point error of the robotic arm.

[0075] In practical applications, the camera module can be calibrated first. For example, it can be calibrated based on the focal length and origin. Then, the calibrated camera module can be used to capture images of the target object at different shooting distances. Based on the obtained images, the correspondence between pixel distance and actual distance can be established for different shooting distances. For example, the principle of similar triangles can be used to determine the correspondence between pixel distance and actual distance, resulting in: Actual distance = Pixel distance × Shooting distance / Pixel distance parameter. Here, the pixel distance parameter can be the pixel distance corresponding to the shooting distance. After obtaining the shooting distance of the needle path image and determining the first pixel distance between the center point of the needle path and the target needle entry point in the needle path image, the first physical distance corresponding to the first pixel distance at that shooting distance can be determined based on the correspondence between pixel distance and actual distance. The pixel distance between the center point of the needle path and the target needle entry point is then converted into the actual distance, and this actual distance is used as the needle entry point error of the robotic arm.

[0076] Figure 3 A schematic diagram of needle path image acquisition is provided. According to... Figure 3The system allows for pre-setting of the target needle insertion point on the target object. After registration, a registration verification process is performed, prompting the user to replace the dedicated registration verification device adapter 120. The device adapter 120 has a wheel mounted on its top, equipped with a laser module and a camera module. After replacement, the wheel can be adjusted to align with the laser module, and the device adapter 120 can be moved to a verification posture. For example, the robotic arm end effector can be moved to the verification point, aligning the needle path on the device adapter 120 with the target needle insertion point. The laser module is then activated, and the laser beam passes through the needle path on the device adapter 120, illuminating the target object and forming a laser point. The user can subjectively determine the needle insertion point error by comparing the position of the laser point with the position of the target needle insertion point. For example, the distance between the laser point and the target needle insertion point can be estimated to obtain the needle insertion point error. Alternatively, the laser module can be used for laser ranging to obtain the distance between the laser point and the origin of the laser module. Next, adjust the wheel to the camera module so that the camera module is facing the needle path on the instrument adapter 120 to take a picture of the target object. Based on the captured needle path image, the error of the needle insertion point is objectively evaluated. In the objective evaluation process, the distance between the laser point and the origin of the laser module can be used as the shooting distance DistA of the needle path image.

[0077] Figure 4 A schematic diagram is provided for determining the needle insertion point error based on needle path images. According to... Figure 4 If the needle path can be a circular hole, then the needle path in the needle path image will be circular, with the center of the circle being the center point of the needle path. The distance from the center of the circle to the edge of the needle path is the radius of the needle path. From the needle path image, the number of pixels that determine the radius of the needle path is r. p And obtain the shooting distance DistA of the needle path image, according to the mapping relationship d2=f(d1,d p ), thus obtaining the needle path radius DistB = f(DistA, r p The camera module can send the needle path image to the control panel, where the user can confirm the image. If the user confirms that the target needle entry point does not exist in the image, it means the needle entry point error is greater than the needle path radius DistB. The control panel will display that the current error is greater than DistB, and re-registration is recommended. Otherwise, if the user confirms that the target needle entry point exists in the image, the pixel distance d between the target needle entry point and the center point of the needle path is determined. p And obtain the shooting distance DistA of the needle path image, according to the mapping relationship d2=f(d1,d p The needle insertion point error DistC = f(DistA,d) is obtained. p ).

[0078] The aforementioned method for determining the needle insertion point error of the robotic arm involves acquiring an image of the target object by photographing the needle path of the instrument adapter with a camera module, and acquiring the shooting distance of the camera module photographing the target object with the needle path. If the needle path image contains the target needle insertion point, the first pixel distance between the center point of the needle path in the needle path image and the target needle insertion point is determined. According to a preset mapping relationship, a first physical distance corresponding to the shooting distance and the first pixel distance is determined, and the first physical distance is determined as the needle insertion point error of the robotic arm. When the captured needle path image contains the target needle insertion point, the center point of the needle path in the image is taken as the actual needle insertion point. The pixel distance between the actual needle insertion point and the target needle insertion point is calculated, and then the pixel distance is converted into a physical distance according to the mapping relationship between the pixel distance and the physical distance to obtain the actual error distance between the actual needle insertion point and the target needle insertion point. Since no puncture needle is needed and no manual measurement is required, the accuracy of determining the needle insertion point error can be improved.

[0079] In one embodiment, prior to step S210, the method may further include: determining the location of a verification point in the medical image based on the location of the target needle insertion point in the medical image of the target object and the location of the target point in the medical image of the target object; determining the location of the verification point in the environment where the robotic arm is located based on the registration matrix of the robotic arm and the location of the verification point in the medical image; and controlling the end effector of the robotic arm to move to the location of the verification point in the environment.

[0080] Among them, medical imaging can be three-dimensional images of the target object.

[0081] The target point can be the point inside the target object that is expected to be reached during the needle insertion operation.

[0082] The verification point can be the point that the robotic arm end effector is expected to reach.

[0083] The registration matrix can be a transformation matrix between the medical image coordinate system and the robotic arm base coordinate system.

[0084] In practice, the controller can determine the target point coordinates and the needle insertion point coordinates in the 3D medical image of the target object. On the extension of the line connecting the target point coordinates and the needle insertion point coordinates, the verification point coordinates are determined. Based on the transformation matrix between the medical image coordinate system and the robotic arm base coordinate system, the verification point coordinates are transformed from the medical image coordinate system to the robotic arm base coordinate system, thus obtaining the coordinates of the verification point in the robotic arm base coordinate system. Then, the controller can control the end effector of the robotic arm to move to the coordinates of the verification point in the robotic arm base coordinate system.

[0085] For example, according to Figure 3In the medical image of the target object, the target point and the target needle insertion point are determined. A line is drawn from the target point to the target needle insertion point, and then extended to a verification point. The coordinates of the verification point in the medical image are converted to the coordinates of the robotic arm base coordinate system. Before determining the needle insertion point error, the end effector of the robotic arm can be moved to the location of the verification point in the robotic arm base coordinate system. The distance between the target needle insertion point and the verification point can be preset, and the position of the verification point can be determined based on this distance. Alternatively, the position of the verification point can be determined manually by adjusting the line connecting the target point and the target needle insertion point. This embodiment of the application does not impose any limitations on this method.

[0086] In this embodiment, the position of the verification point is determined in the medical image based on the position of the target needle insertion point in the medical image and the position of the target point in the medical image. Based on the registration matrix of the robotic arm and the position of the verification point in the medical image, the position of the verification point in the environment where the robotic arm is located is determined. By controlling the end effector of the robotic arm to move to the position of the verification point in the environment, the end effector of the robotic arm can be aligned with the target needle insertion point as much as possible, the needle insertion point error can be minimized, and the accuracy of the robot performing the needle insertion operation can be increased.

[0087] In one embodiment, after step S210, the method may further include: if the needle path image does not contain the target needle entry point, determining the second pixel distance between the center point of the needle path and the edge line of the needle path in the needle path image; determining the second physical distance corresponding to the shooting distance and the second pixel distance according to the mapping relationship, and determining that the needle entry point error is greater than the second physical distance.

[0088] The second physical distance can be the actual distance between the center point of the needle path and the edge line of the needle path.

[0089] In practice, the controller can determine the pixel distance r between the center point of the needle track and the edge line of the needle track in the needle track image. p And obtain the shooting distance DistA of the needle track image, and obtain the second physical distance DistB = f(DistA, r) according to the pre-set mapping relationship. p The controller can also detect whether there is a target needle entry point in the needle path image. If no target needle entry point is detected, it means that the distance between the actual needle entry point and the target needle entry point will exceed the second physical distance, and the needle entry point error is determined to be greater than the second physical distance.

[0090] For example, according to Figure 4 The controller identifies the needle path radius in the needle path image as 5 pixels and determines DistA = 10cm. According to the mapping relationship, the actual needle path radius DistB = f(10cm, 5 pixels) = 0.01cm. If the target needle entry point is not detected in the needle path image, it can be determined that the needle entry point error is greater than 0.01cm.

[0091] In this embodiment, when the needle path image does not contain the target needle entry point, the second pixel distance between the center point of the needle path and the edge line of the needle path in the needle path image is determined; according to the mapping relationship, the second physical distance corresponding to the shooting distance and the second pixel distance is determined, and it is determined that the needle entry point error is greater than the second physical distance. This allows for rapid and quantitative determination of the needle entry point error when the needle entry point error is large and the target needle entry point cannot be seen within the camera's field of view, thereby improving the efficiency of needle entry point error determination.

[0092] In one embodiment, after step S210, the method may further include: displaying a needle path image; receiving a first confirmation message or a second confirmation message for the needle path image; the first confirmation message indicating that the needle path image contains a target needle entry point, and the second confirmation message indicating that the needle path image does not contain a target needle entry point.

[0093] In practice, the presence of a target needle insertion point in the needle path image can also be manually detected. The controller can display the needle path image, and the user can identify the displayed needle path image. If the target needle insertion point is identified in the needle path image, a first confirmation message is input to the controller; if the target needle insertion point is not identified in the needle path image, a second confirmation message is input to the controller.

[0094] In this embodiment, by displaying a needle path image and receiving a first confirmation message or a second confirmation message for the needle path image, the presence of a target needle insertion point in the needle path image can be determined through human-computer interaction, thereby improving the accuracy of the judgment on the presence of a target needle insertion point.

[0095] In one embodiment, upon receiving a second confirmation message, the method may further include: displaying at least one candidate needle insertion point in the needle path image; and determining a target needle insertion point in the needle path image in response to a selection operation for the at least one candidate needle insertion point.

[0096] Among them, the candidate needle insertion point can be the target needle insertion point to be verified.

[0097] In the specific implementation, if the controller receives the second confirmation message, it means that the needle path image contains the target needle entry point. The controller can display at least one candidate needle entry point in the needle path image. The user can select a target candidate needle entry point from at least one candidate needle entry point, perform a selection operation on the target candidate needle entry point, and determine it as the target needle entry point.

[0098] For example, when a user determines that the needle path image contains a target needle entry point, the user inputs a second confirmation message to the controller. When the controller receives the second confirmation message, it can display all candidate needle entry points A, B, C, D, and E in the needle path image. The user selects the target needle entry point A from these, and the controller will calculate the needle entry point error for the target needle entry point A.

[0099] In this embodiment, by displaying at least one candidate needle entry point in the needle path image, and in response to the selection operation for at least one candidate needle entry point, the target needle entry point in the needle path image is determined. The target needle entry point that needs to be determined for error determination can be determined through human-computer interaction, thereby further improving the accuracy of needle entry point error determination.

[0100] In one embodiment, before step S210, the method may further include: acquiring the laser point generated by the laser module emitting laser from the needle path to the target object; if the distance between the laser point and the target needle entry point is less than a preset threshold, then acquiring the needle path image obtained by the camera module taking a picture of the target object with the needle path facing the instrument adapter, in order to determine the needle entry point error of the robotic arm.

[0101] The laser module can be, but is not limited to, various laser emitters. The laser module can communicate with the controller via wired or wireless links.

[0102] In a specific implementation, a laser module can be installed on the instrument adapter. The laser module can emit a laser from the needle path of the instrument adapter towards the target object, generating a laser point on the target object. The controller can collect the position of the laser point and compare the position of the laser point with the position of the target needle insertion point. If the distance between the laser point and the target needle insertion point is greater than or equal to a preset threshold, it indicates that the needle insertion point error is large and re-registration is required. Otherwise, if the distance between the laser point and the target needle insertion point is less than the preset threshold, the needle insertion point error determination method of the robotic arm in the above embodiment of this application is executed. Since the needle insertion point error determination method of the robotic arm has been described in detail in the foregoing embodiments, it will not be repeated here.

[0103] In practical applications, a wheel can be installed on the top of the instrument adapter, equipped with a laser module and a camera module. The wheel is adjusted to align with the laser module, and the instrument adapter is moved to a verification posture. For example, the robotic arm's end effector can be moved to the verification point, with the needle path on the instrument adapter facing the target insertion point. The laser module is then switched on, and the laser beam passes through the needle path onto the target object, forming a laser dot. The controller can identify the position of the laser dot and the target insertion point, calculating the actual distance between them. Alternatively, the actual distance can be manually measured and input into the controller. The controller then compares the measured distance with a preset threshold. If the distance falls within the threshold range... If the error exceeds the preset threshold, it indicates a large needle insertion point error, requiring re-registration. Otherwise, if the actual distance between the laser point and the target needle insertion point does not exceed the preset threshold, it indicates a small needle insertion point error. The error can be further determined by the camera module. In this case, the wheel can be adjusted to the camera module, and the controller acquires the needle path image obtained by the camera module shooting the target object towards the needle path of the instrument adapter, as well as the shooting distance of the camera module shooting the target object towards the needle path. If the needle path image contains the target needle insertion point, the first pixel distance between the center point of the needle path in the needle path image and the target needle insertion point is determined. According to the preset mapping relationship, the first physical distance corresponding to the shooting distance and the first pixel distance is determined, and the first physical distance is determined as the needle insertion point error of the robotic arm.

[0104] In this embodiment, the laser module emits a laser from the needle path towards the target object, generating a laser point. If the distance between the laser point and the target needle entry point is less than a preset threshold, the camera module takes a picture of the target object from the needle path of the instrument adapter, and obtains a needle path image to determine the needle entry point error of the robotic arm. This allows the laser module to make a preliminary judgment before using the camera module to determine the needle entry point error. If the needle entry point error is not large, the camera module can be used to further determine the needle entry point error, thus improving the efficiency of needle entry point error determination.

[0105] In one embodiment, step S210 may specifically include: obtaining the laser ranging distance between the laser module and the laser point; and determining the shooting distance at which the camera module takes pictures of the target object towards the needle track based on the positional relationship between the laser module and the camera module, and the laser ranging distance.

[0106] Among them, the laser ranging distance can be the actual distance between the origin of the laser module and the laser point.

[0107] In practice, a laser module and a camera module can be installed sequentially on the instrument adapter. The spatial relationship between the origin of the laser module and the origin of the camera module can be calculated. The laser module on the instrument adapter can also be used to emit a laser from the needle path towards the target object to perform laser ranging and obtain the actual distance between the origin of the laser module and the laser point on the target object. The controller can obtain the spatial relationship between the origin of the laser module and the origin of the camera module, as well as the actual distance between the origin of the laser module and the laser point. Based on the spatial relationship and the actual distance between the origin of the laser module and the laser point, the shooting distance for the camera module to shoot the target object towards the needle path can be obtained.

[0108] In practical applications, the wheel on the instrument adapter can be adjusted to the laser module first to obtain the spatial coordinates of the laser module's origin. Then, the wheel can be adjusted to the camera module to obtain the spatial coordinates of the camera module's origin. Based on the relative positions of the laser module's origin and the camera module's origin, the spatial relationship between them can be determined. Alternatively, the wheel can be adjusted to the laser module to emit laser light from the needle path towards the target needle insertion point on the target object for laser ranging. The actual distance between the laser module's origin and the laser point can be obtained. By adjusting the actual distance between the laser module's origin and the laser point based on the spatial relationship, the shooting distance of the needle path image can be obtained.

[0109] In this embodiment, by obtaining the laser ranging distance between the laser module and the laser point, and based on the positional relationship between the laser module and the camera module, as well as the laser ranging distance, the shooting distance of the camera module towards the needle path to photograph the target object is determined. This enables the efficient and accurate determination of the shooting distance of the camera module towards the needle path to photograph the target object, improving the efficiency and accuracy of needle entry point error determination.

[0110] To facilitate a deeper understanding of the embodiments of this application by those skilled in the art, a specific example will be used for illustration below.

[0111] To address the issues of operator subjectivity in the registration and verification process of neurosurgical robots, and the deformation of puncture needles due to prolonged use, which leads to inaccurate error measurements, a camera or laser module can be installed on the top of the instrument adapter to align with the needle path, creating a new dedicated end effector for registration and verification. The method for determining the needle insertion point error may include the following steps:

[0112] 1. Before leaving the factory, the camera module needs to be calibrated internally to obtain the distortion parameters, focal length, origin coordinates, pixel distance, and physical distance of the camera module. Based on the above parameters, the correspondence between pixel distance and actual distance is established for different shooting distances. It is also possible to calibrate the spatial position relationship between the origin of the camera module and the origin of the laser module for the camera module and laser module respectively installed on the top of the instrument adapter.

[0113] 2. Registration verification begins. The user is prompted to replace the dedicated registration verification device adapter and install the laser module. The user then moves to the verification posture, turns on the laser switch, and the laser shines onto the head through a small hole. The user can subjectively judge the magnitude of the error with their naked eye. After the subjective judgment that the error is acceptable, the distance between the laser point and the origin of the laser module is measured. Then, the camera module is replaced for objective error assessment. At this point, the distance between the target needle insertion point and the origin of the camera module can be calculated based on the spatial relationship between the origins of the laser module and the camera module, and recorded as the shooting distance.

[0114] 3. Obtain the needle track image through the circular hole. Based on the correspondence between pixel distance and actual distance and the shooting distance, the distance between the center point of the needle track and the edge line of the needle track (circular hole radius) can be calculated and recorded as the second physical distance. The second physical distance can also be directly obtained by using mechanical processing parameters and camera intrinsic parameters as known data.

[0115] 4. Upload the needle path image to the Console for display. Ask the user to confirm whether the target needle entry point exists in the needle path image. If it does not exist, the needle entry point error is greater than the second physical distance, and the user will be shown that the current error is greater than the second physical distance, and it is recommended to re-register. If it exists, in order to prevent multiple target needle entry points from appearing in the hole, ask the user to select the target needle entry point. By using the correspondence between pixel distance and actual distance and the shooting distance, the distance between the position of the target needle entry point clicked by the user in the needle path image and the center point of the needle path image can be calculated and recorded as the first physical distance, which is the actual needle entry point error value.

[0116] The aforementioned method for determining needle insertion point error employs an instrument adapter that can accommodate both a laser module and a camera module. The laser module emits a laser beam from the needle path of the instrument adapter towards the target object. Based on the laser point formed on the target object, the user can visually perceive the distance between this laser point and the target needle insertion point on the object, i.e., the needle insertion point error. Simultaneously, a needle path image is obtained by using laser ranging and the camera module to capture the target object through the needle path of the instrument adapter. If the target needle insertion point exists in the needle path image, the needle insertion point error is calculated based on the pixel distance between the center point of the needle path and the target needle insertion point in the needle path image, as well as the physical distance between the camera module and the target needle insertion point. If the target needle insertion point does not exist in the needle path image, the needle insertion point error is determined to be greater than the needle path radius. This method simplifies the workflow, eliminating the need for the user to physically contact the needle insertion point with the puncture needle. It also eliminates concerns about needles being too short to reach the point or bending due to prolonged use; the needle insertion point error can be calculated simply by performing a click operation on the image.

[0117] Moreover, by using a combination of laser and camera modules and employing a subjective and objective evaluation method, the system is not affected by the inconsistency of reference values ​​caused by the user's subjective readings. It also provides a superior subjective evaluation method using laser irradiation compared to traditional puncture needles.

[0118] Furthermore, existing technologies often rely on rulers or human visual judgment to estimate the needle entry point error. The software is unaware of this value and therefore cannot make decisions or provide recommendations based on it. The needle entry point error calculated from the image eliminates the need for ruler or visual estimation. Simultaneously, the software is aware of this value and can use it to provide operational suggestions to the user, and even offer robotic arm path execution compensation suggestions based on multiple sets of needle entry point errors.

[0119] In one embodiment, such as Figure 5 As shown, a method for determining the needle entry point error of a robotic arm is provided, which is then applied to... Figure 1 Taking controller 140 as an example, the explanation includes the following steps:

[0120] Step S310: Based on the position of the target needle insertion point in the medical image of the target object and the position of the target point in the medical image, determine the position of the verification point in the medical image; based on the registration matrix of the robotic arm and the position of the verification point in the medical image, determine the position of the verification point in the environment where the robotic arm is located; control the end effector of the robotic arm to move to the position of the verification point in the environment.

[0121] Step S320: Obtain the needle track image obtained by the camera module shooting the target object with the needle track facing the instrument adapter, and obtain the shooting distance of the camera module shooting the target object with the needle track facing the needle track;

[0122] Step S330: Display the needle path image; receive a first confirmation message or a second confirmation message for the needle path image;

[0123] Step S340: If a first confirmation message is received, determine the first pixel distance between the center point of the needle path in the needle path image and the target needle entry point; determine the first physical distance corresponding to the shooting distance and the first pixel distance according to the preset mapping relationship, and determine the first physical distance as the needle entry point error of the robotic arm.

[0124] Step S350: If a second confirmation message is received, determine the second pixel distance between the center point of the needle path and the edge line of the needle path in the needle path image; determine the second physical distance corresponding to the shooting distance and the second pixel distance according to the mapping relationship, and determine that the needle entry point error is greater than the second physical distance.

[0125] Since the controller's processing procedure has been described in detail in the foregoing embodiments, it will not be repeated here.

[0126] The aforementioned method for determining the needle insertion point error of the robotic arm involves: determining the position of the verification point in the medical image based on the position of the target needle insertion point in the medical image and the position of the target point in the medical image; determining the position of the verification point in the environment where the robotic arm is located based on the registration matrix of the robotic arm and the position of the verification point in the medical image; controlling the end effector of the robotic arm to move to the position of the verification point in the environment; acquiring a needle path image of the target object with the camera module facing the needle path of the instrument adapter, and acquiring the shooting distance of the camera module facing the needle path of the target object; displaying the needle path image; receiving a first confirmation message or a second confirmation message for the needle path image; if the first confirmation message is received, then... The process involves determining a first pixel distance between the center point of the needle path in the needle path image and the target needle insertion point; determining a first physical distance corresponding to the shooting distance and the first pixel distance based on a preset mapping relationship, and defining this first physical distance as the needle insertion point error of the robotic arm; if a second confirmation message is received, determining a second pixel distance between the center point of the needle path in the needle path image and the edge line of the needle path; determining a second physical distance corresponding to the shooting distance and the second pixel distance based on the mapping relationship, and determining that the needle insertion point error is greater than the second physical distance; by allowing the user to confirm whether a target needle insertion point exists in the needle path image, the needle insertion point error can be determined for different target needle insertion point situations, facilitating human-computer interaction and further improving the efficiency of needle insertion point error determination. Moreover, since no puncture needle is needed and no manual measurement is required, the accuracy of needle insertion point error determination can be improved.

[0127] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0128] In one embodiment, such as Figure 6 As shown, a system 400 for determining the needle entry point error of a robotic arm is provided, including an instrument adapter 120 and a controller 140; the instrument adapter 120 is equipped with a camera module;

[0129] The camera module is used to take a picture of the target object towards the needle path of the instrument adapter 120, obtain a needle path image, and send the needle path image to the controller 140;

[0130] The controller 140 is used to acquire a needle track image obtained by the camera module shooting the target object towards the needle track of the instrument adapter, and to acquire the shooting distance of the camera module shooting the target object towards the needle track; if the needle track image contains the target needle entry point, then determine the first pixel distance between the center point of the needle track in the needle track image and the target needle entry point; according to a preset mapping relationship, determine the first physical distance corresponding to the shooting distance and the first pixel distance, and determine the first physical distance as the needle entry point error of the robotic arm.

[0131] In one embodiment, the controller 140 is further configured to determine the position of a verification point in the medical image based on the position of the target needle insertion point in the medical image of the target object and the position of the target point in the medical image of the target object; determine the position of the verification point in the environment where the robotic arm is located based on the registration matrix of the robotic arm and the position of the verification point in the medical image; and control the end effector of the robotic arm to move to the position of the verification point in the environment.

[0132] In one embodiment, the controller 140 is further configured to: if the needle path image does not contain the target needle insertion point, determine a second pixel distance between the center point of the needle path in the needle path image and the edge line of the needle path; determine a second physical distance corresponding to the shooting distance and the second pixel distance according to the mapping relationship; and determine that the needle insertion point error is greater than the second physical distance.

[0133] In one embodiment, the controller 140 is further configured to display the needle path image; receive a first confirmation message or a second confirmation message for the needle path image; the first confirmation message indicates that the needle path image contains the target needle insertion point, and the second confirmation message indicates that the needle path image does not contain the target needle insertion point.

[0134] In one embodiment, the controller 140 is further configured to display at least one candidate needle insertion point in the needle path image; and to determine a target needle insertion point in the needle path image in response to a selection operation for the at least one candidate needle insertion point.

[0135] In one embodiment, the instrument adapter 120 is provided with a laser module; the laser module is used to emit a laser from the needle path toward the target object to generate a laser spot.

[0136] In one embodiment, the controller 140 is further configured to perform a step of acquiring a needle track image of the target object with the camera module facing the instrument adapter based on a trigger command, so as to determine the needle entry point error of the robotic arm; wherein the trigger command is generated when the distance between the laser point and the target needle entry point is less than a preset threshold.

[0137] In one embodiment, the laser module is further configured to determine the laser ranging distance between the laser module and the laser point, and send the laser ranging distance to the controller 140; the controller 140 is further configured to receive the laser ranging distance, and determine the shooting distance at which the camera module shoots the target object towards the needle track based on the positional relationship between the laser module and the camera module and the laser ranging distance.

[0138] In one embodiment, such as Figure 7 As shown, a surgical robot 500 is provided, including a robotic arm 600 and a needle entry point error determination system 400 for the robotic arm. The needle entry point error determination system 400 is used to implement the steps in the above-described method embodiments.

[0139] The aforementioned robotic arm needle entry point error determination system and surgical robot use a camera module on the instrument adapter to capture images of the target object along the needle path of the instrument adapter, obtaining a needle path image. This image is then sent to the controller. The controller acquires the needle path image captured by the camera module along the needle path of the instrument adapter, as well as the shooting distance of the camera module. If the needle path image contains the target needle entry point, the first pixel distance between the center point of the needle path in the image and the target needle entry point is determined. Based on a preset mapping relationship, a first physical distance corresponding to the shooting distance and the first pixel distance is determined, and this first physical distance is defined as the needle entry point error of the robotic arm. When the captured needle path image contains the target needle entry point, the center point of the needle path in the image is taken as the actual needle entry point. The pixel distance between the actual needle entry point and the target needle entry point is calculated, and then the pixel distance is converted into a physical distance based on the mapping relationship between pixel distance and physical distance, yielding the actual error distance between the actual needle entry point and the target needle entry point. Since no puncture needle is required and no manual measurement is needed, the accuracy of needle entry point error determination can be improved.

[0140] Based on the same inventive concept, this application also provides a device for determining the needle entry point error of a robotic arm, which is used to implement the above-described method for determining the needle entry point error of a robotic arm. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the device for determining the needle entry point error of a robotic arm provided below can be found in the limitations of the method for determining the needle entry point error of a robotic arm described above, and will not be repeated here.

[0141] In one embodiment, a device for determining the needle entry point error of a robotic arm is provided, comprising: an acquisition module, a calculation module, and a determination module, wherein:

[0142] The acquisition module is used to acquire the needle track image obtained by the camera module taking pictures of the target object with the needle track facing the instrument adapter, and to acquire the shooting distance of the camera module taking pictures of the target object with the needle track facing the needle track;

[0143] A calculation module is used to determine a first pixel distance between the center point of the needle path in the needle path image and the target needle entry point if the needle path image contains the target needle entry point;

[0144] The determination module is used to determine a first physical distance corresponding to the shooting distance and the first pixel distance according to a preset mapping relationship, and to determine the first physical distance as the needle entry point error of the robotic arm.

[0145] In one embodiment, the above-mentioned device for determining the needle entry point error of the robotic arm further includes:

[0146] The location determination module is used to determine the location of the verification point in the medical image based on the location of the target needle insertion point in the medical image of the target object and the location of the target point of the target object in the medical image;

[0147] A position conversion module is used to determine the position of the verification point in the environment where the robotic arm is located, based on the registration matrix of the robotic arm and the position of the verification point in the medical image.

[0148] A motion control module is used to control the end effector of the robotic arm to move to the position of the verification point in the environment.

[0149] In one embodiment, the above-mentioned device for determining the needle entry point error of the robotic arm further includes:

[0150] The second pixel distance determination module is used to determine the second pixel distance between the center point of the needle path in the needle path image and the edge line of the needle path if the needle path image does not contain the target needle insertion point.

[0151] The second physical distance determination module is used to determine a second physical distance corresponding to the shooting distance and the second pixel distance according to the mapping relationship, and to determine that the needle entry point error is greater than the second physical distance.

[0152] In one embodiment, the above-mentioned device for determining the needle entry point error of the robotic arm further includes:

[0153] Acupuncture path image display module is used to display the acupuncture path image;

[0154] A confirmation message receiving module is used to receive a first confirmation message or a second confirmation message for the needle path image; the first confirmation message is used to indicate that the needle path image contains the target needle insertion point, and the second confirmation message is used to indicate that the needle path image does not contain the target needle insertion point.

[0155] In one embodiment, the above-mentioned device for determining the needle entry point error of the robotic arm further includes:

[0156] The candidate needle entry point display module is used to display at least one candidate needle entry point in the needle path image;

[0157] A target needle insertion point determination module is used to determine a target needle insertion point in the needle path image in response to a selection operation for the at least one candidate needle insertion point.

[0158] In one embodiment, the above-mentioned device for determining the needle entry point error of the robotic arm further includes:

[0159] The laser point acquisition module is used to acquire the laser point generated by the laser module emitting laser from the needle path to the target object;

[0160] The distance judgment module is used to determine the needle entry point error of the robotic arm by taking a picture of the target object with the camera module facing the instrument adapter if the distance between the laser point and the target needle entry point is less than a preset threshold.

[0161] In one embodiment, the acquisition module is further configured to acquire the laser ranging distance between the laser module and the laser point; and determine the shooting distance at which the camera module takes pictures of the target object towards the needle track based on the positional relationship between the laser module and the camera module and the laser ranging distance.

[0162] Each module in the aforementioned device for determining the needle entry point error of the robotic arm can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0163] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for determining the needle insertion point error of a robotic arm. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0164] Those skilled in the art will understand that Figure 8The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0165] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0166] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0167] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0168] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for determining the needle entry point error of a robotic arm, characterized in that, The method includes: The camera module captures an image of the target object by pointing towards the needle path of the instrument adapter, and the camera module captures the target object at the shooting distance of the needle path. If the needle path image contains a target needle entry point, then the first pixel distance between the center point of the needle path in the needle path image and the target needle entry point is determined. According to a preset mapping relationship, the first physical distance corresponding to the shooting distance and the first pixel distance is determined, and the first physical distance is determined as the needle entry point error of the robotic arm. If the needle path image does not contain the target needle entry point, then the second pixel distance between the center point of the needle path in the needle path image and the edge line of the needle path is determined. Based on the mapping relationship, the second physical distance corresponding to the first physical distance shooting distance and the second pixel distance is determined, and the needle entry point error is determined to be greater than the second physical distance.

2. The method according to claim 1, characterized in that, Before acquiring the needle track image obtained by the camera module facing the needle track of the instrument adapter, the following steps are also included: Based on the location of the target needle insertion point in the medical image of the target object, and the location of the target point in the medical image of the target object, the location of the verification point is determined in the medical image; Based on the registration matrix of the robotic arm and the position of the verification point in the medical image, the position of the verification point in the environment where the robotic arm is located is determined. Control the end effector of the robotic arm to move to the location of the verification point in the environment.

3. The method according to claim 1, characterized in that, The method further includes: Display the needle path image; Receive a first confirmation message or a second confirmation message for the needle path image; the first confirmation message is used to indicate that the needle path image contains the target needle insertion point, and the second confirmation message is used to indicate that the needle path image does not contain the target needle insertion point.

4. The method according to claim 3, characterized in that, Upon receiving the second confirmation message, the method further includes: Display at least one candidate needle entry point in the needle path image; In response to the selection operation for the at least one candidate needle entry point, a target needle entry point in the needle path image is determined.

5. The method according to claim 1, characterized in that, Before acquiring the needle track image obtained by the camera module facing the needle track of the instrument adapter, the following steps are also included: The laser module emits laser light from the needle path toward the target object, generating laser spots; If the distance between the laser point and the target needle insertion point is less than a preset threshold, then the step of acquiring the needle path image of the target object with the camera module facing the instrument adapter is executed to determine the needle insertion point error of the robotic arm.

6. The method according to claim 5, characterized in that, The step of obtaining the shooting distance at which the camera module shoots the target object towards the needle path includes: Obtain the laser ranging distance between the laser module and the laser point; Based on the positional relationship between the laser module and the camera module, and the laser ranging distance, the shooting distance at which the camera module shoots the target object towards the needle path is determined.

7. A system for determining the needle entry point error of a robotic arm, characterized in that, The system includes an instrument adapter and a controller; the instrument adapter is equipped with a camera module. The camera module is used to take a picture of the target object towards the needle path of the instrument adapter, obtain a needle path image, and send the needle path image to the controller; The controller is used to perform the method according to any one of claims 1 to 6.

8. The system according to claim 7, characterized in that, The instrument adapter is equipped with a laser module; The laser module is used to emit laser light from the needle path toward the target object to generate laser points.

9. The system according to claim 8, characterized in that, The controller is also used to execute a step of acquiring a needle track image of the target object by the camera module facing the instrument adapter based on a trigger command, so as to determine the needle entry point error of the robotic arm; wherein the trigger command is generated when the distance between the laser point and the target needle entry point is less than a preset threshold.

10. The system according to claim 8, characterized in that, The laser module is also used to determine the laser ranging distance between the laser module and the laser point, and to send the laser ranging distance to the controller; The controller is further configured to receive the laser ranging distance, and determine the shooting distance at which the camera module takes pictures of the target object towards the needle track based on the positional relationship between the laser module and the camera module, and the laser ranging distance.

11. A surgical robot, characterized in that, The surgical robot includes a robotic arm and the system according to any one of claims 7 to 10.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images