Verification method, verification system, computer device and readable storage medium

Abstract

The application relates to a verification method, a verification system, a computer device and a readable storage medium. The method comprises the following steps: determining first position information of a verification point from a medical image of a target object; determining second position information of a reference point from the medical image based on the verification point; the reference point is a point on a puncture path passing through the verification point, the puncture path is perpendicular to the surface of the target object, or the puncture path is an axial path of a puncture marker corresponding to the verification point; and the first position information and the second position information are used to control the movement of a mechanical arm to verify the registration information of the mechanical arm. By using the method, the verification and judgment of two dimensions of position and direction can be realized, so that the accuracy of the registration information verification can be ensured, and the precision and accuracy of the registration verification are improved.

Description

Technical Field

[0001] This application relates to the field of medical technology, and in particular to a verification method, verification system, computer equipment, and readable storage medium. Background Technology

[0002] In the medical field, for surgical robots, when performing surgical operations using surgical robots, it is usually necessary to spatially register the robotic arm, that is, to determine the registration matrix between the robotic arm's spatial coordinate system and the image coordinate system. In order to ensure the accuracy of the registration matrix, it is usually necessary to verify the registration matrix based on one or more verification points.

[0003] However, the traditional method of verifying the registration matrix through verification points suffers from low verification accuracy. Summary of the Invention

[0004] Therefore, it is necessary to provide a verification method, verification device, verification system, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of registration verification in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a verification method, including:

[0006] Determine the primary location information of the verification point from the medical images of the target object;

[0007] The second location information of the reference point is determined from the medical image based on the verification point; the reference point is a point on the puncture path that passes through the verification point, the puncture path is perpendicular to the surface of the target object, or the puncture path is the axial path of the puncture marker corresponding to the verification point.

[0008] The robotic arm is controlled to move based on the first and second position information in order to verify the registration information of the robotic arm.

[0009] In one embodiment, the verification point includes feature points of the target object, and second location information for determining reference points from the medical image based on the verification point includes:

[0010] Identify the tangent plane that passes through the verification point and is tangent to the surface of the target object from medical images;

[0011] The target vertical line that is perpendicular to the cutting plane and passes through the verification point is determined as the puncture path;

[0012] A reference point is determined along the puncture path, and secondary location information of the reference point is determined from medical images.

[0013] In one embodiment, the verification point includes the vertex of a puncture marker set on the target object, and second location information for determining a reference point from the medical image based on the verification point includes:

[0014] Identify puncture markers at the verification point from medical images;

[0015] The axis along which the puncture marker is located is defined as the puncture path;

[0016] A reference point is determined along the puncture path, and secondary location information of the reference point is determined from medical images.

[0017] In one embodiment, the method further includes:

[0018] In response to the verification point switching command sent by the switching device on the robotic arm, the step of determining the first position information of the next verification point from the medical image is executed.

[0019] In one embodiment, after determining the first location information of the verification point from the medical image of the target object, the method further includes:

[0020] Based on the first location information of the verification point, the verification point in the medical image is marked; the marking process includes at least one of highlighting and adding marking information.

[0021] The labeled medical images are displayed on the screen.

[0022] In one embodiment, the medical image includes a three-dimensional medical image. After determining the first location information of the verification point from the medical image of the target object, the method further includes:

[0023] Based on the first location information of the verification point, the viewing angle of the three-dimensional medical image is adjusted so that the verification point is included in the preset display area of ​​the display screen;

[0024] The three-dimensional medical image with the adjusted viewing angle is displayed on the screen.

[0025] In one embodiment, the medical image includes multiple reconstructed planar images corresponding to different orientations. After determining the first location information of the verification point from the medical image of the target object, the method further includes:

[0026] For each orientation, based on the first location information of the verification point, the target plane reconstruction image including the verification point is determined from multiple plane reconstruction images corresponding to the orientation.

[0027] The target plane reconstruction images are displayed on the screen in all directions.

[0028] Secondly, this application also provides a verification device, comprising:

[0029] The first determining module is used to determine the first location information of the verification point from the medical image of the target object;

[0030] The second determining module is used to determine the second location information of the reference point from the medical image based on the verification point; the reference point is a point on the puncture path that passes through the verification point, the puncture path is perpendicular to the surface of the target object, or the puncture path is the axial path of the puncture marker corresponding to the verification point.

[0031] The verification module is used to control the movement of the robotic arm based on the first position information and the second position information to verify the registration information of the robotic arm.

[0032] Thirdly, this application also provides a verification system, which includes verification equipment and a robotic arm;

[0033] The verification device is used to perform the steps of the verification method in the first aspect above to verify the registration information of the robotic arm.

[0034] Fourthly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the verification method in the first aspect described above.

[0035] Fifthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the verification method described in the first aspect above.

[0036] Sixthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the verification method described in the first aspect above.

[0037] The aforementioned verification method, verification device, verification system, computer equipment, storage medium, and computer program product determine a first location information of a verification point from a medical image of a target object, and a second location information of a reference point from the medical image based on the verification point; then, based on the first and second location information, control the movement of a robotic arm to verify the registration information of the robotic arm; wherein, the reference point is a point on the puncture path passing through the verification point, the puncture path being perpendicular to the surface of the target object, or the puncture path being the axial path of the puncture marker corresponding to the verification point. In other words, using the method proposed in this application embodiment, another reference point corresponding to the verification point can be determined on an observable and referable puncture path, so that the end-effector posture of the robotic arm determined by the verification point and the reference point matches the observable and referable puncture path, thereby facilitating the user to verify the directional dimension of the registration information based on the difference between the actual posture of the robotic arm end-effector and the observable and referable puncture path; since this verification method can determine both position and direction, it can ensure the accuracy of registration information verification, thereby improving the precision and accuracy of registration verification. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a diagram illustrating the application environment of the verification method in one embodiment;

[0040] Figure 2 This is a flowchart illustrating the verification method in one embodiment;

[0041] Figure 3 This is a flowchart illustrating the verification method in another embodiment;

[0042] Figure 4 This is a flowchart illustrating the verification method in another embodiment;

[0043] Figure 5 This is a flowchart illustrating the verification method in another embodiment;

[0044] Figure 6 This is a flowchart illustrating the verification method in another embodiment;

[0045] Figure 7 This is a flowchart illustrating the verification method in another embodiment;

[0046] Figure 8 This is a schematic diagram of the complete process of the verification method in one embodiment;

[0047] Figure 9(a) is a schematic diagram of the verification point operation interface in one embodiment;

[0048] Figure 9(b) is a schematic diagram of the autonomous motion interface of the robotic arm in one embodiment;

[0049] Figure 9(c) is a schematic diagram of the adapter installation interface in one embodiment;

[0050] Figure 9(d) is a schematic diagram of the verification point switching interface in one embodiment;

[0051] Figure 10 This is a structural block diagram of the verification device in one embodiment;

[0052] Figure 11 This is a schematic diagram of the verification system in one embodiment;

[0053] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] In the medical field, for surgical robots, spatial registration of the robot's robotic arm is typically required before surgical procedures can be performed. For example, after completing contact-based (marked, such as bone screw) spatial registration, a neurosurgical robot obtains a registration matrix from the robotic arm's spatial coordinate system to the image coordinate system. To ensure the accuracy of the registration matrix, it is validated using verification points.

[0056] In other words, during verification, a verification point is selected, and then the robotic arm is automatically controlled based on the registration matrix to aim for the end effector of the robotic arm to reach the position corresponding to the verification point. Next, an adapter is installed, and a verification probe is used to test whether the end effector of the robotic arm accurately moves to the position corresponding to the selected verification point, that is, whether the tip of the verification probe can move to the verification point, to verify whether the registration matrix is ​​accurate.

[0057] Traditionally, after selecting a verification point, users need to manually adjust the Virtual Reality (VR) window in the image window to their own perspective for easy observation of the verification point on the VR image. The VR window displays a 3D medical image of the target object. Users then manually navigate to the corresponding verification point in the Multiplanar Reconstruction (MPR) window, which displays tomographic images in three different orientations: transverse, coronal, and sagittal. Furthermore, there are no special prompts in the VR and MPR windows to identify the verification point, making it difficult for users to easily identify the current verification point when there are many verifications.

[0058] Furthermore, during conventional verification, the system only forms a verification path based on the coordinates of a single verification point and the coordinates of the volume data center point of the image, and then uses the registration matrix to control the automatic movement of the robotic arm. Since the reference point selected by traditional verification methods is the center point of the volume data, the direction of the probe cannot accurately match the direction of the verification path corresponding to the current verification point when using verification probes to verify accuracy, leading to unreliable verification accuracy. Moreover, traditional verification methods can also encounter scenarios where the orientation of the verification point and the reference point is abnormal, causing the robotic arm to fail to move autonomously to the designated position for that verification point. In other words, the end-effector pose of the robotic arm determined by the verification point and the volume data center point exceeds the robotic arm's range of motion, preventing the robotic arm from reaching that end-effector pose and thus making it impossible to verify that verification point.

[0059] Furthermore, in traditional verification methods, when multiple verification points are involved, users need to manually select the next verification point on the system's software interface after verifying one verification point, which is quite cumbersome for users.

[0060] To address the multiple technical problems existing in the above-mentioned traditional verification methods, this application proposes a verification method that fundamentally solves the problem of low verification accuracy in traditional verification methods; on this basis, it can also solve the other technical problems mentioned above.

[0061] The verification method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, the verification device 102 can be a computer device equipped with an imaging system, or other computer devices such as a verification terminal or verification server. For example, the verification device can be equipped with a verification system that can communicate with the robotic arm that needs to be verified. The verification system can also display the patient's medical images, and can also support users to mark verification points in the medical images, or automatically identify markers in the medical images as verification points.

[0062] In one exemplary embodiment, such as Figure 2 As shown, a verification method is provided, which is applied to... Figure 1 The following steps are used as an example of the verification equipment: steps 202 to 206. Wherein:

[0063] Step 202: Determine the first location information of the verification point from the medical image of the target object.

[0064] The verification point can be a feature point of the target object, such as facial feature points, including but not limited to points with significant characteristics such as the tip of the nose, eyeballs, lips, and eyebrows. It can also be a point on a marker pasted or fixed to the target object, such as the apex of a bone nail fixed to the skull of the target object. It should be noted that the bone nail used can be of any form or structure, and the apex structure of different bone nails may differ; for example, if the top of the bone nail has a groove structure, the verification point can be the center point of the largest cross-section of the groove at the top of the bone nail. In this embodiment, the form and location of the verification point are not specifically limited. Furthermore, the target object can be a phantom, or a living human or animal. When the target object is a phantom, it can be used for experimental verification or simulated operation scenarios.

[0065] For example, if there is a marker on the target object, the medical image obtained after scanning the target object may include the marker. In this case, the verification device can identify the marker in the medical image to determine the verification point in the medical image corresponding to the marker on the target object, and then obtain the first position information of the verification point from the medical image. The first position information of the verification point may be the coordinate information in the image coordinate system of the medical image.

[0066] For example, the verification device can also determine the verification point of the target object by identifying and extracting the feature points of the target object in the medical image, thereby determining the first location information of the verification point from the medical image.

[0067] Step 204: Determine the second location information of the reference point from the medical image based on the verification point.

[0068] The reference point is a point on the puncture path that passes through the verification point. The puncture path is perpendicular to the surface of the target object. Alternatively, the puncture path is the axial path of the puncture marker corresponding to the verification point.

[0069] For example, when the first location information of the verification point is determined from the medical image, the puncture path passing through the verification point can be determined based on the first location information of the verification point; then, any point other than the verification point is determined from the puncture path as the reference point, so as to determine the second location information of the reference point from the medical image.

[0070] The first location information of the verification point and the second location information of the reference point can both be coordinate information in the image coordinate system.

[0071] For example, the reference point can be any point on the puncture path pointing from the verification point into the target object, or any point on the puncture path pointing from the verification point out of the target object; this application embodiment does not specifically limit this. Preferably, when the end-effector posture of the robotic arm is determined based on the vector pointing from the reference point to the verification point, selecting any point on the puncture path pointing from the verification point out of the target object as the reference point ensures that the probe at the end of the robotic arm will not cause damage to the target object during its movement from outside the target object (e.g., above the verification point) to the verification point or its vicinity, thereby improving the safety and reliability of the verification.

[0072] In other words, assuming any point along the path from the verification point to the target object is chosen as the reference point, when determining the end effector posture of the robotic arm based on the vector from the reference point to the verification point, the end effector position of the robotic arm may be below the verification point (i.e., below the target object's body), and the probe's trajectory under the end effector posture is from below the verification point to the verification point. In this process, the probe needs to pierce the target object from below its body to reach the verification point, thereby causing damage to the target object.

[0073] For example, the verification device can first determine from the medical image whether a puncture marker corresponding to the verification point exists. If a puncture marker corresponding to the verification point exists in the medical image, the axis of the puncture marker can be determined, and the axis of the puncture marker can be determined as the puncture path passing through the verification point. If no puncture marker corresponding to the verification point exists in the medical image, an axis passing through the verification point and perpendicular to the surface of the target object can be determined as the puncture path passing through the verification point.

[0074] Step 206: Control the movement of the robotic arm based on the first position information and the second position information to verify the registration information of the robotic arm.

[0075] In one implementation, when both the first and second position information are coordinate information in the image coordinate system, the verification device can perform coordinate transformation on the first and second position information according to the registration information (such as the registration matrix) to obtain the third position information of the verification point and the fourth position information of the reference point in the robot arm coordinate system. Then, the third position information of the verification point and the fourth position information of the reference point are sent to the robot arm to control the robot arm to move. After the robot arm moves to the corresponding position, the registration information of the robot arm is verified.

[0076] In another implementation, the verification device can also send the first position information of the verification point and the second position information of the reference point in the image coordinate system to the robotic arm, so as to instruct the robotic arm to perform coordinate transformation on the first position information and the second position information according to the registration information (such as the registration matrix) to obtain the third position information of the verification point and the fourth position information of the reference point in the robotic arm coordinate system. Then, the robotic arm moves according to the third position information of the verification point and the fourth position information of the reference point, and verifies the registration information of the robotic arm after it moves to the corresponding position.

[0077] For example, when verifying registration information, the user can mount a verification probe or a visible light device (such as a laser) on the end of a robotic arm to observe whether the verification probe or laser can reach the verification point, and at the same time observe whether the path of the verification probe or laser is perpendicular to the surface of the target object, or whether it coincides with the axis of the puncture marker; by verifying the puncture end and puncture direction at the same time, the accuracy of the registration information can be verified.

[0078] For example, the verification device can also use other auxiliary devices (such as vision sensors) to verify the accuracy of the robotic arm's registration information by checking whether the position of the verification probe or laser on the target object's surface is consistent with the position of the verification point, whether the puncture direction is perpendicular to the target object's surface, or whether it coincides with the axis of the puncture marker. The specific method of registration verification is not limited in this application embodiment.

[0079] In the above verification method, a first position information of the verification point is determined from the medical image of the target object, and a second position information of the reference point is determined from the medical image based on the verification point. Then, the robotic arm is controlled to move based on the first and second position information to verify the registration information of the robotic arm. The reference point is a point on the puncture path passing through the verification point, and the puncture path is perpendicular to the surface of the target object; alternatively, the puncture path is the axial path of the puncture marker corresponding to the verification point. In other words, the method proposed in this application can determine another reference point corresponding to the verification point on an observable and referable puncture path, so that the end-effector posture of the robotic arm determined by the verification point and the reference point matches the observable and referable puncture path. This allows users to verify the directional dimension of the registration information based on the difference between the actual posture of the robotic arm end-effector and the observable and referable puncture path. Since this verification method can determine both position and direction, it can ensure the accuracy of registration information verification, thereby improving the precision and accuracy of registration verification.

[0080] In an exemplary embodiment, the verification point may include feature points of the target object or vertices of puncture markers set on the target object. Different implementation methods may be used to determine the second position information of the reference point for different types of verification points. The following will provide the corresponding methods for determining the reference point for the two types of verification points.

[0081] The first type: when the verification points include feature points of the target object, such as... Figure 3 As shown, step 204 above may include steps 302 to 306. Wherein:

[0082] Step 302: Determine the tangent plane from the medical image that passes through the verification point and is tangent to the surface of the target object.

[0083] For example, the medical image can be a three-dimensional medical image of the target object, such as: a three-dimensional reconstruction can be performed based on a scanned image of the target object to obtain a three-dimensional medical image of the target object; optionally, during the reconstruction process, the surface of the target object can be reconstructed to obtain a three-dimensional medical image containing the surface of the target object; for example, in the field of neurosurgery, the medical image can be a three-dimensional reconstructed image of the head.

[0084] For example, for this medical image, the verification device can obtain one or more verification points by identifying feature points of the target object in the medical image; for example, feature points on the head and face, including but not limited to points with distinctive characteristics such as the tip of the nose, eyeballs, lips, and eyebrows. Then, after determining the verification points, for the currently selected verification point, a tangent plane that passes through the verification point and is tangent to the surface of the target object can be determined in the medical image; since the surface of the target object can be similar to a curved surface, determining the tangent plane corresponding to the verification point is equivalent to determining the tangent plane of a certain point on the curved surface.

[0085] Step 304: Determine the target vertical line that is perpendicular to the cutting plane and passes through the verification point as the puncture path.

[0086] There are usually many perpendicular lines to a plane, but if a point on the plane is determined, then there is one and only one perpendicular line passing through that point and perpendicular to the plane. In this implementation, when the slice plane corresponding to the verification point is determined, the target perpendicular line passing through the verification point and perpendicular to the slice plane is determined as the puncture path passing through the verification point.

[0087] Step 306: Determine a reference point from the puncture path and determine the second location information of the reference point from the medical image.

[0088] The puncture path passing through the verification point can be divided into two parts by the verification point: one part is the path from the verification point towards the interior of the target object, and the other part is the path from the verification point towards the exterior of the target object. According to standard robotic arm usage, the end effector is typically moved to a position above the verification point. Therefore, in this embodiment, any point can be arbitrarily determined as a reference point on the portion of the puncture path outside the target object, based on the distance between the robotic arm end effector and the verification point. For example, a point at a distance from the verification point on the portion of the puncture path outside the target object can be determined as another reference point corresponding to the verification point. Furthermore, the second position information of this reference point can be determined from the medical image.

[0089] The second type: when the verification point includes the vertex of the puncture marker set on the target object, such as... Figure 4 As shown, step 204 above may include steps 402 to 406. Wherein:

[0090] Step 402: Identify the puncture markers where the verification point is located from the medical images.

[0091] The puncture markers may include, but are not limited to, bone screws fixed to the surface of the target object, or any type of marker pasted to the surface of the target object.

[0092] For example, the verification device can identify puncture markers in medical images to obtain one or more candidate puncture markers, and then determine the puncture marker where the verification point is located based on the first location information of the verification point and the location information of each candidate puncture marker.

[0093] Step 404: Determine the axis where the puncture marker is located as the puncture path.

[0094] For example, when the verification device identifies the puncture marker at the verification point, it can determine the centerline of the puncture marker along its height direction. This centerline is the axis of the puncture marker, which is also the puncture path passing through the verification point. The height direction of the puncture marker can be a direction pointing inwards from the puncture marker to the target object. In practical applications, the puncture marker (such as a bone screw) can be fixed vertically to the surface of the target object, in which case the height direction of the bone screw is perpendicular to the target surface; or it can be fixed at a certain angle to the surface of the target object, in which case the height direction of the bone screw has an angle with the target object surface, rather than being perpendicular, but its height direction still points inwards from the target object.

[0095] Step 406: Determine a reference point from the puncture path and determine the second location information of the reference point from the medical images.

[0096] Please refer to the relevant description in step 306 above; it will not be repeated here.

[0097] The above embodiments describe the methods for determining reference points corresponding to two different types of verification points. In actual application scenarios, there may be other types of verification points, and their implementation methods can refer to the above two methods to determine the puncture path that passes through the verification point and has certain reference value, and then determine the reference point on the puncture path to improve the accuracy of registration verification.

[0098] In an exemplary embodiment, a verification point switching scheme is proposed to address the problem that users need to switch verification points on the display interface in traditional technologies, which leads to low user operation convenience. In this embodiment, a switching device can be set on a robotic arm, and the switching device is communicatively connected to the verification device. By triggering the switching device on the robotic arm, the user can control the verification device to switch to the next verification point. Correspondingly, the verification device can respond to the verification point switching command sent by the switching device on the robotic arm and perform the step of determining the first position information of the next verification point from the medical image.

[0099] For example, the switching device can be set at various joint positions of the robotic arm, such as the end joint of the robotic arm. After the user completes the verification of the registration information through the end, he / she can directly trigger the switching device on the end, so that the verification device can control the robotic arm to move to the vicinity of the next verification point and continue to verify the registration information.

[0100] For example, the switching device can be any structure such as a button, knob, or lever, and the embodiments of this application do not specifically limit it.

[0101] Using the method in this embodiment, after completing verification at one verification point, the user can directly trigger the switching device on the robotic arm to automatically jump to the next verification point. This reduces human-computer interaction with the display interface, allowing the user to focus more on the robotic arm's movement. This not only improves the convenience of switching verification points but also increases operational efficiency, thereby improving registration verification efficiency.

[0102] In one exemplary embodiment, to address the problem in conventional technologies where verification points are not specially marked, leading to difficulties in quickly and accurately determining the current verification point when multiple verification points exist, a verification point marking scheme is proposed; such as... Figure 5 As shown, after step 202 above, the method may further include:

[0103] Step 502: Mark the verification points in the medical image according to the first location information of the verification points.

[0104] The marking process may include at least one of highlighting or adding marking information; the marking information may include, but is not limited to, marking symbols, marking text, marking patterns, etc.

[0105] In this embodiment, after determining the current verification point and its first location information in the medical image, the verification device can mark the current verification point. For example, the color attribute, transparency attribute, brightness attribute, etc. of the current verification point can be adjusted to achieve the effect of highlighting the verification point; or, marking information can be added to the current verification point to highlight the current verification point in the medical image and achieve the effect of making the verification point eye-catching.

[0106] Step 504: Display the labeled medical image on the display screen.

[0107] For example, after marking the verification points in the medical image, the verification device can render the marked medical image on the display screen; optionally, the display screen can be the display screen of the verification device, or an extended display screen connected to the verification device, etc.

[0108] In this embodiment, by marking the verification points currently being verified in the medical image, the user can immediately identify the verification points through the marked medical image displayed on the screen. This not only improves the display effect of the verification points but also enhances the user experience, thereby increasing verification efficiency and shortening the verification time.

[0109] In an exemplary embodiment, to address the problem in conventional technologies where users need to manually adjust the VR window of the image window to their own viewpoint after selecting a verification point, resulting in cumbersome and inefficient operation, an automatic VR viewpoint adjustment scheme is proposed; in cases where medical images include 3D medical images, such as Figure 6 As shown, after step 202 above, the method may further include:

[0110] Step 602: Based on the first position information of the verification point, adjust the viewing angle of the three-dimensional medical image so that the verification point is included in the preset display area of ​​the display screen.

[0111] During registration verification, it is usually necessary to verify multiple verification points at different locations to ensure accuracy and precision. However, when multiple verification points are located at different positions on the target object, it may not be possible to display multiple verification points simultaneously from a single viewpoint of a 3D medical image; for example, when verification points include the apex of a bone screw at the back of the head and the tip of the nose on the face, it is impossible to observe both verification points simultaneously from one viewpoint.

[0112] Therefore, when verifying different verification points, it is necessary to adjust the perspective of the 3D medical image to ensure that the user can observe the verification point on the display interface.

[0113] For example, in this embodiment, after determining the first location information of the verification point, the verification device can also adjust the viewing angle of the three-dimensional medical image according to the first location information, so that the verification point moves to a preset display area of ​​the display screen; wherein, the preset display area can be the central area of ​​the display screen, or other display areas of the display screen customized by the user, etc.

[0114] In other words, when verifying different verification points, the viewing angle of the 3D medical image is automatically adjusted to ensure that the verification point is always located in the center area of ​​the display screen for easy viewing by the user.

[0115] Step 604: Display the adjusted 3D medical image on the display screen.

[0116] In this embodiment, by automatically adjusting the perspective of the three-dimensional medical image, the user can avoid the process of manually adjusting the image perspective every time they switch verification points. This not only improves the efficiency of image perspective adjustment but also enhances the user experience, thereby improving verification efficiency and shortening verification time.

[0117] In an exemplary embodiment, to address the problem in traditional techniques where users need to manually flip through layers in the MPR window to obtain the corresponding layer image after selecting a verification point, resulting in cumbersome and inefficient operation, an automatic MPR layer image selection scheme is proposed. This scheme is applicable when medical images include multiple planar reconstructed images corresponding to different orientations, such as... Figure 7 As shown, after step 202 above, the method may further include:

[0118] Step 702: For each orientation, based on the first position information of the verification point, determine the target plane reconstruction image that includes the verification point from multiple plane reconstruction images corresponding to the orientation.

[0119] The MPR window can display tomographic images in three different orientations: transverse, coronal, and sagittal. For each orientation, multi-plane reconstructed images can be obtained by processing the three-dimensional medical images, which means multiple tomographic images for each orientation.

[0120] When verifying verification points at different locations, the target plane reconstruction image corresponding to each verification point can be selected from the multi-plane reconstruction images in each direction based on the first location information of the verification point. That is, the target plane reconstruction image corresponding to the verification point in the transverse position, the target plane reconstruction image corresponding to the verification point in the coronal position, and the target plane reconstruction image corresponding to the verification point in the sagittal position can be selected.

[0121] Step 704: Display the target plane reconstruction images in all directions on the display screen.

[0122] In other words, the MPR window on the display screen displays the target plane reconstruction image corresponding to the verification point in the transverse plane, the target plane reconstruction image corresponding to the verification point in the coronal plane, and the target plane reconstruction image corresponding to the verification point in the sagittal plane.

[0123] In this embodiment, by automatically filtering the MPR image from all angles, the user can avoid the process of manually selecting the tomographic image corresponding to the verification point each time the verification point is switched. This not only improves the efficiency of adjusting the image perspective but also enhances the user experience, thereby improving verification efficiency and shortening the verification time.

[0124] In one exemplary embodiment, such as Figure 8As shown, a complete implementation process for registration verification is provided, including the following steps:

[0125] The first step is to enter the space registration verification process. The first verification point is selected by default, but you can also select other verification points at this time.

[0126] The selected verification point is highlighted and automatically performs VR and MPR positioning. The selected verification point also supports dragging, fine-tuning (panning in the up, down, left, and right directions), showing / hiding, undoing, redoing, and resetting (returning to the initial position). Referring to Figure 9(a), in the image display area, the upper left corner displays the 3D medical image after viewpoint adjustment, with the verification point located in the center display area and marked with a circle. The lower left, upper right, and lower right corners are the target plane reconstruction images from three different directions, and the verification point is also marked with a circle in each of these reconstructed images. Additionally, the right-hand display area of ​​the screen displays other information, including multiple verification points V1, V2, V3, and V4, with V1 currently displayed; it also includes verification accuracy information.

[0127] The second step is for the robotic arm to move autonomously.

[0128] Before the robotic arm begins autonomous movement, the verification device can automatically calculate the current verification point and another reference point corresponding to that verification point. For example, it can determine the puncture path passing through the verification point based on the verification point, and then determine the reference point above the verification point based on the default path length. Next, the verification device can assign the verification point and reference point to the robotic arm to automatically position it in the corresponding orientation. When the user steps on the foot pedal, the robotic arm autonomously moves to the position corresponding to the verification point, as shown in Figure 9(b).

[0129] From the perspective of operational safety, after the verification equipment sends the verification point and reference point to the robotic arm, the robotic arm does not move immediately, but moves only after the user gives a movement instruction; for example, the robotic arm is instructed to move after the user steps on the foot pedal.

[0130] The third step is to determine whether the verification points meet the verification requirements. Install the adapter, as shown in Figure 9(c). Use the verification probe to verify whether the robotic arm moves accurately to the corresponding verification point, thereby verifying the accuracy of the registration matrix.

[0131] Fourth step, proceed to the next verification point. Press the confirmation button at the end of the robotic arm, and it will automatically proceed to the next verification point for verification, as shown in Figure 9(d).

[0132] In the above verification interaction process, for the selected verification point, the corresponding verification point on the medical image can be highlighted, and the outer frame of the verification point can be indicated with circles of different colors. At the same time, the verification point VR window is automatically positioned in the user's view, and the MPR window is switched to the maximum diameter level of the current point. This makes it easier for the user to identify the current verification point and saves doctors the trouble of finding the selected verification point by flipping through layers and adjusting the window position.

[0133] In addition, before the robotic arm starts moving autonomously at the selected verification point, the current verification point and the corresponding reference point are automatically calculated. The verification point and the reference point are then assigned to the robotic arm to automatically position it in the corresponding direction. This is more in line with the actual path execution motion scenario, thus enabling more accurate verification of the registration matrix accuracy.

[0134] Furthermore, because a confirmation button is integrated into the end effector of the robotic arm, pressing the confirmation button after completing one verification point will automatically jump to the next verification point. This reduces human-computer interaction with the software interface, allowing users to focus more on the robotic arm's movement and resulting in more efficient operation.

[0135] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0136] Based on the same inventive concept, this application also provides a verification apparatus for implementing the verification method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more verification apparatus embodiments provided below can be found in the limitations of the verification method described above, and will not be repeated here.

[0137] In one exemplary embodiment, such as Figure 10 As shown, a verification device is provided, including: a first determining module 1002, a second determining module 1004, and a verification module 1006, wherein:

[0138] The first determining module 1002 is used to determine the first location information of the verification point from the medical image of the target object.

[0139] The second determining module 1004 is used to determine the second position information of the reference point from the medical image based on the verification point; the reference point is a point on the puncture path that passes through the verification point, the puncture path is perpendicular to the surface of the target object, or the puncture path is the axial path of the puncture marker corresponding to the verification point.

[0140] The verification module 1006 is used to control the movement of the robotic arm based on the first position information and the second position information to verify the registration information of the robotic arm.

[0141] In one embodiment, the verification points include feature points of the target object, and the second determining module 1004 includes:

[0142] The first determining unit is used to determine, from the medical image, a tangent plane that has passed the verification point and is tangent to the surface of the target object;

[0143] The second determining unit is used to determine the target vertical line that is perpendicular to the cutting plane and passes through the verification point as the puncture path;

[0144] The third determining unit is used to determine the reference point from the puncture path and to determine the second location information of the reference point from the medical image.

[0145] In one embodiment, the verification point includes the vertex of a puncture marker set on the target object, and the second determining module 1004 includes:

[0146] The identification unit is used to identify the puncture markers where the verification point is located from medical images;

[0147] The fourth determining unit is used to determine the axis where the puncture marker is located as the puncture path;

[0148] The fifth determining unit is used to determine the reference point from the puncture path and to determine the second location information of the reference point from the medical image.

[0149] In one embodiment, the device further includes:

[0150] The switching module is used to respond to the verification point switching command sent by the switching device set on the robotic arm and to perform the step of determining the first position information of the next verification point from the medical image.

[0151] In one embodiment, the device further includes:

[0152] The marking module is used to mark the verification points in the medical image after the first determining module 1002 determines the first location information of the verification points in the medical image; the marking process includes at least one of highlighting and adding marking information.

[0153] The first display module is used to display the labeled medical images on the display screen.

[0154] In one embodiment, the medical imaging includes three-dimensional medical imaging, and the device further includes:

[0155] The adjustment module is used to adjust the viewing angle of the three-dimensional medical image after the first determining module 1002 determines the first position information of the verification point from the medical image, so that the verification point is included in the preset display area of ​​the display screen.

[0156] The second display module is used to display the three-dimensional medical images after the viewing angle has been adjusted on the display screen.

[0157] In one embodiment, the medical images include multiple reconstructed planar images corresponding to different orientations, and the device further includes:

[0158] The third determining module is used to determine the target plane reconstruction image including the verification point in each direction based on the first location information of the verification point from multiple plane reconstruction images corresponding to the direction after the first determining module 1002 determines the first location information of the verification point in the medical image.

[0159] The third display module is used to display the target plane reconstruction images in all directions on the display screen.

[0160] Each module in the aforementioned verification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0161] In one exemplary embodiment, a verification system is provided, such as Figure 11 As shown, the verification system includes a verification device 102 and a robotic arm 104; the verification device 102 and the robotic arm 104 are communicatively connected. The verification device 102 is used to execute the steps of the verification method in any of the above embodiments to verify the registration information of the robotic arm 104.

[0162] For example, the verification system may also include a switching device (not shown in the figure) mounted on the robotic arm 104, and the verification device 102 is communicatively connected to the switching device. The verification device 102 is used to receive a verification point switching instruction sent by the switching device and to perform the step of determining the first position information of the next verification point from the medical image.

[0163] When verifying the registration information of the robotic arm using this verification system, the implementation method can be found in the relevant descriptions of the verification methods in the above embodiments, and will not be repeated here.

[0164] In one exemplary embodiment, a computer device is provided, which may be a verification device. When the verification device is a terminal, its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a verification method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0165] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0166] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the verification method in any of the above embodiments.

[0167] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the verification method in any of the above embodiments.

[0168] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the verification method in any of the above embodiments.

[0169] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0170] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0171] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A verification method, characterized in that, The method includes: Determine the primary location information of the verification point from the medical images of the target object; The second location information of the reference point is determined from the medical image based on the verification point; the reference point is a point on the puncture path passing through the verification point, and is any point on the puncture path in the direction of the path from the verification point to the outside of the target object, the puncture path is perpendicular to the surface of the target object, or the puncture path is the axial path of the puncture marker corresponding to the verification point. Based on the first position information and the second position information, the robotic arm is controlled to move in order to verify the registration information of the robotic arm; The verification point includes feature points of the target object. The second location information of the reference point determined from the medical image based on the verification point includes: determining a cutting plane that passes through the verification point and is tangent to the surface of the target object from the medical image; determining a target vertical line that is perpendicular to the cutting plane and passes through the verification point as the puncture path; determining a reference point from the puncture path; and determining the second location information of the reference point from the medical image. Alternatively, the verification point may include the vertex of a puncture marker set on the target object, and the second location information of the reference point determined from the medical image based on the verification point may include: identifying the puncture marker where the verification point is located from the medical image; determining the axis where the puncture marker is located as the puncture path; determining the reference point from the puncture path, and determining the second location information of the reference point from the medical image.

2. The method according to claim 1, characterized in that, The method further includes: In response to a verification point switching command sent by a switching device mounted on the robotic arm, the step of determining the first position information of the next verification point from the medical image is executed.

3. The method according to claim 1, characterized in that, After determining the first location information of the verification point from the medical image of the target object, the method further includes: Based on the first location information of the verification point, the verification point in the medical image is marked; the marking process includes at least one of highlighting and adding marking information. The labeled medical images are displayed on the screen.

4. The method according to claim 1, characterized in that, The medical image includes a three-dimensional medical image. After determining the first location information of the verification point from the medical image of the target object, the method further includes: Based on the first location information of the verification point, the viewing angle of the three-dimensional medical image is adjusted so that the verification point is included in the preset display area of ​​the display screen; The three-dimensional medical image with the viewing angle adjusted is displayed on the display screen.

5. The method according to claim 1, characterized in that, The medical images include multiple reconstructed planar images corresponding to different orientations. After determining the first location information of the verification point from the medical images of the target object, the method further includes: For each of the aforementioned orientations, based on the first location information of the verification point, a target plane reconstruction image including the verification point in the orientation is determined from multiple plane reconstruction images corresponding to the orientation. The reconstructed images of the target plane at each of the aforementioned orientations are displayed on the display screen.

6. A verification system, characterized in that, The verification system includes verification equipment and a robotic arm; The verification device is used to perform the steps of the method as described in any one of claims 1 to 5, to verify the registration information of the robotic arm.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images