Methods, devices, surgical navigation systems, and media for calibrating coordinate transformation relationships.

Abstract

This invention discloses a method, apparatus, surgical navigation system, and medium for calibrating coordinate transformation relationships. The method includes: adjusting the end effector of a robotic arm to a first target position; rigidly connecting a calibration plate to the base of the robotic arm; and ensuring that the heights of at least four preset markers correspond to the heights of at least four key points on the body surface of a corresponding human region. The method also involves acquiring the first spatial positions of the at least four preset markers in a lidar coordinate system and the second spatial positions of the at least four preset markers in the base coordinate system corresponding to the robotic arm; and calibrating the mapping relationship between the lidar coordinate system and the end effector coordinate system using the first spatial positions of the at least four preset markers in the lidar coordinate system and the third spatial positions in the end effector coordinate system. This invention solves the problem of low accuracy in lidar-based surgical navigation, improves the accuracy of lidar for surgical navigation, and reduces the cost of surgical navigation systems.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to methods, apparatus, surgical navigation systems and media for calibrating coordinate transformation relationships. Background Technology

[0002] Current surgical navigation relies on structured light for surgical positioning. However, structured light is easily affected by ambient lighting, such as the operating room lamp, which impacts the accuracy of surgical navigation. In contrast, lidar is less affected by ambient lighting. However, due to the precision limitations of lidar technology, both the lidar coordinate system and the surgical navigation system's navigation coordinate system need to be calibrated before they can be used for positioning during surgery. Therefore, lidar-based surgical navigation suffers from accuracy limitations. Summary of the Invention

[0003] This invention provides a method, apparatus, surgical navigation system, and medium for calibrating coordinate transformation relationships, in order to solve the problem of low accuracy in lidar-based surgical navigation.

[0004] According to one aspect of the present invention, a method for calibrating coordinate transformation relationships is provided, the method comprising:

[0005] The end effector of the robotic arm is adjusted to the first target position so that the distance between the end effector of the robotic arm and the calibration plate is the set calibration distance. At least four preset markers are set on the calibration plate. The calibration plate is rigidly connected to the base of the robotic arm. The height of the at least four preset markers corresponds to the height of at least four key points on the body surface of the corresponding human body area.

[0006] The first spatial positions of at least four preset markers in the lidar coordinate system and the second spatial positions of at least four preset markers in the base coordinate system corresponding to the robotic arm are obtained, wherein the lidar is set at the end of the robotic arm and the mapping relationship between the base coordinate system and the end coordinate system of the robotic arm is known.

[0007] The mapping relationship between the lidar coordinate system and the terminal coordinate system is calibrated by taking the first spatial position of at least four preset markers in the lidar coordinate system as independent variables and the third spatial position of at least four preset markers in the terminal coordinate system as dependent variables.

[0008] According to another aspect of the present invention, a calibration device for coordinate transformation relationships is provided, the device comprising:

[0009] The robotic arm control module is used to adjust the end effector of the robotic arm to the first target position so that the distance between the end effector of the robotic arm and the calibration plate is the set calibration distance. At least four preset markers are set on the calibration plate. The calibration plate is rigidly connected to the base of the robotic arm. The height of the at least four preset markers corresponds to the height of at least four key points on the body surface of the corresponding human body area.

[0010] The spatial position acquisition module is used to acquire the first spatial position of at least four preset markers in the lidar coordinate system and the second spatial position of at least four preset markers in the base coordinate system corresponding to the robotic arm, wherein the lidar is set at the end of the robotic arm and the mapping relationship between the base coordinate system and the end coordinate system of the robotic arm is known.

[0011] The mapping relationship calibration module is used to calibrate the mapping relationship between the lidar coordinate system and the end coordinate system, with the first spatial position of at least four preset markers in the lidar coordinate system as independent variables and the third spatial position of at least four preset markers in the end coordinate system as dependent variables.

[0012] According to another aspect of the present invention, a surgical navigation system is provided, the system comprising:

[0013] A robotic arm, the end of which is used to hold surgical instruments;

[0014] The calibration plate is equipped with at least four preset markers and is rigidly connected to the base of the robotic arm. The height of the at least four preset markers corresponds to the height of at least four key points on the body surface of the corresponding human body region.

[0015] A lidar device, mounted at the end of a robotic arm, is used to acquire the first spatial positions of at least four preset markers in a lidar coordinate system.

[0016] At least one processor; and

[0017] A memory that is communicatively connected to at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to obtain the second spatial positions of at least four preset markers in the base coordinate system of the robotic arm and to execute the calibration method of coordinate transformation relationship according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement a calibration method for coordinate transformation relationships according to any embodiment of the present invention.

[0020] The technical solution of this invention calibrates the mapping relationship between the lidar coordinate system and the end-effector coordinate system by obtaining the spatial position of the preset marker in the lidar coordinate system and the base coordinate system of the robotic arm, thereby improving the accuracy of lidar for surgical navigation and reducing the cost of the surgical navigation system.

[0021] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily apparent from the following description. The acquisition, storage, use, and processing of data in the technical solutions of this application all comply with the relevant provisions of national laws and regulations. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a structural block diagram of a surgical navigation system provided according to an embodiment of the present invention;

[0024] Figure 2 This is a flowchart of a coordinate transformation relationship calibration method provided by an embodiment of the present invention;

[0025] Figure 3 This is a flowchart of another method for calibrating coordinate transformation relationships according to an embodiment of the present invention;

[0026] Figure 4A This is a structural block diagram of a coordinate transformation relationship calibration device provided according to an embodiment of the present invention;

[0027] Figure 4B This is a structural block diagram of a calibration device for another coordinate transformation relationship provided by an embodiment of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," "third," "fourth," and "fifth," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] Figure 1 This is a structural block diagram of a surgical navigation system provided according to an embodiment of the present invention. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0031] like Figure 1 As shown, the surgical navigation system 10 includes a robotic arm 4, the end of which is used to fix surgical instruments; a calibration plate 5, which is configured with at least four preset markers and is rigidly connected to the base of the robotic arm 4, the height of the at least four preset markers corresponding to the height of at least four key points on the body surface of the corresponding human body area; a lidar device 6, which is disposed at the end of the robotic arm 4 and is used to obtain the first spatial position of the at least four preset markers in the lidar coordinate system; at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11.

[0032] In surgical procedures, especially in neurosurgical navigation, an accuracy of less than 1 mm at the surgical site on the body surface is sufficient to meet the surgical requirements. Although the average spatial positioning accuracy of lidar is at the centimeter and sub-centimeter level, the accuracy range within the target space can be limited by a calibration plate. In other words, the position of the target object during each surgery is within the set calibration distance range, so that the positioning accuracy of the target space meets the surgical requirements.

[0033] In one embodiment, the preset marker is a cone-shaped marker. To simulate the undulations of areas such as the face / abdomen, multiple cone-shaped markers are set, with a number greater than or equal to four. These cone-shaped markers are placed on a calibration plate 5, which is rigidly connected to the base of the robotic arm 4. This ensures that the position of the calibration plate 5 relative to the base of the robotic arm 4 does not change during the LiDAR device 6's imaging process and the surgical procedure; that is, the actual spatial position of the cone-shaped marker in the base coordinate system of the robotic arm 4 does not change. Furthermore, since the actual spatial position of the cone-shaped marker in the base coordinate system does not change, the third spatial position of the cone-shaped marker in the end-effector coordinate system is determined based on the current pose of the end-effector of the robotic arm 4 corresponding to the actual spatial position. The first spatial position of the cone-shaped marker in the LiDAR coordinate system is then calibrated, thereby calibrating the coordinate transformation relationship between the LiDAR coordinate system of the target space corresponding to the operating table and the end-effector coordinate system of the robotic arm.

[0034] Understandably, multiple cone-shaped markers can also be placed. This improves the accuracy and precision of the coordinate transformation between the LiDAR coordinate system and the robotic arm coordinate system. Furthermore, at least four preset markers, corresponding to the heights of at least four key points on the human body surface, can be placed at predetermined positions on the calibration plate based on the surface undulations of the target object's body region. This simulates the surface undulations of the corresponding area of ​​the target object. For example, this could be an abdominal calibration plate, a back calibration plate, or a facial calibration plate. Specifically, a facial calibration plate can have at least four cone-shaped markers placed in the center to simulate the tip of the nose and eyes of the target object; preset markers of different heights can be designed based on the undulations of the target object's back to serve as a back calibration plate; and preset markers of various heights can be placed at corresponding positions on the calibration plate 5 based on the height of the target object's abdominal undulations to serve as an abdominal calibration plate.

[0035] In one embodiment, the calibration plate 5 and the base of the robotic arm 4 are detachably rigidly connected, and the calibration plate can be disassembled after the coordinate transformation relationship is calibrated.

[0036] The memory stores computer programs executable by at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. The RAM 13 can also store various programs and data required for the operation of the surgical navigation system. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14. Multiple components of the surgical navigation system 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard or mouse; an output unit 17, such as various types of displays or speakers; a storage unit 18, such as a disk or optical disk; and a communication unit 19, such as a network card, modem, or wireless transceiver. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0037] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 is capable of acquiring the second spatial positions of at least four preset markers in the base coordinate system of the robotic arm and performing a calibration method for coordinate transformation relationships described below.

[0038] Figure 2 This is a flowchart of a coordinate transformation relationship calibration method provided by an embodiment of the present invention. This embodiment can be applied to surgical navigation scenarios based on a lidar coordinate system and a robotic arm. The method is configured in the processor of the surgical navigation system.

[0039] like Figure 2 As shown, a method for calibrating coordinate transformation relationships includes the following steps:

[0040] S210. Adjust the end effector of the robotic arm to the first target position so that the distance between the end effector of the robotic arm and the calibration plate is the set calibration distance. At least four preset markers are set on the calibration plate. The calibration plate is rigidly connected to the base of the robotic arm. The height of the at least four preset markers corresponds to the height of at least four key points on the body surface of the corresponding human body area.

[0041] To use the lidar mounted on the end effector of the robotic arm for surgical navigation, the distance between the lidar and the calibration plate needs to be set as the calibration distance. The mapping relationship between the lidar coordinate system and the end effector coordinate system of the robotic arm within this calibration distance needs to be calibrated. Therefore, to calibrate the coordinate transformation relationship between the end effector coordinate system and the lidar coordinate system within the distance range between the end effector and the calibration plate, the end effector of the robotic arm needs to be adjusted to the first target position so that the distance between the end effector and the calibration plate is the set calibration distance.

[0042] When the vertical distance between the end effector of the robotic arm, located directly above the calibration plate, and the calibration plate is the set calibration distance, the position of the end effector is the first target position. For example, this could be the coordinates of the end effector in the world coordinate system when the vertical distance between the end effector and the calibration plate is 28-30 cm.

[0043] S220. Obtain the first spatial positions of at least four preset markers in the lidar coordinate system and the second spatial positions of at least four preset markers in the base coordinate system corresponding to the robotic arm, wherein the lidar is set at the end of the robotic arm and the mapping relationship between the base coordinate system and the end coordinate system of the robotic arm is known.

[0044] Wherein, the first spatial position is the coordinate of the preset marker in the lidar coordinate system. For example, the first spatial position is a point cloud dataset obtained by scanning the preset marker with a 3D lidar device at the end of the robotic arm, where each point in the point cloud dataset contains the 3D coordinates of the preset marker in the lidar coordinate system.

[0045] The second spatial position is the spatial position of the preset marker in the base coordinate system of the robotic arm. Since the preset marker is set on the calibration plate, and the calibration plate is rigidly connected to the base of the robotic arm, the relative positional relationship between the preset marker and the base of the robotic arm will not change. For example, the second spatial position is the three-dimensional coordinate of the preset marker in the base coordinate system of the robotic arm.

[0046] Furthermore, the third spatial positions of at least four preset markers in the end-effector coordinate system of the robotic arm are obtained through the following steps:

[0047] Step a1: Obtain the positional relationship of at least four preset markers relative to the base and the position of the preset known position points in the base coordinate system.

[0048] Here, the known position point is a point on the robot arm base, whose position in the base coordinate system is known and will not change. The known position point, whose position in the base coordinate system will not change, can also be a point on the robot arm base. For example, since the calibration plate and the robot arm base are rigidly connected, the relative position of the preset marker and any point on the robot arm base remains unchanged. Furthermore, by determining the positional relationship between the known position point and the preset marker, the three-dimensional coordinates of the preset marker in the base coordinate system can be obtained.

[0049] Step a2: Based on the known positions and relationships of the known points in the base coordinate system and the known mapping relationship between the end-effector coordinate system and the base coordinate system corresponding to the current pose of the robotic arm, determine the third spatial positions of at least four preset markers in the end-effector coordinate system.

[0050] First, determine the distance between the fixed point rigidly connected to the preset marker on the base and the known position point. Based on the distance from the known position point to the origin of the base coordinate system and the distance between the fixed point and the known position point, determine the coordinates of the preset marker in the base coordinate system. Based on the known mapping relationship between the end effector coordinate system corresponding to the current pose of the robotic arm and the base coordinate system, transform the coordinates of the preset marker in the base coordinate system to the end effector coordinate system through the known mapping relationship to obtain the coordinates of the preset marker in the end effector coordinate system, which is used as the third spatial position.

[0051] For example, the three-dimensional coordinates of the preset marker in the base coordinate system are determined based on the distance between the fixed point rigidly connected to the preset marker on the base and the set known position point and the three-dimensional coordinates of the set known position point in the base coordinate system. The three-dimensional coordinates of the preset marker in the end-effector coordinate system are determined based on the known mapping relationship between the end-effector coordinate system corresponding to the current pose of the robotic arm and the base coordinate system.

[0052] S230. Using the first spatial positions of at least four preset markers in the lidar coordinate system as independent variables and the third spatial positions of at least four preset markers in the end coordinate system as dependent variables, the mapping relationship between the lidar coordinate system and the end coordinate system is calibrated.

[0053] Existing algorithms, programs, or tools that calculate the coordinate transformation matrix between two coordinate systems based on their corresponding coordinates can be used to solve the coordinate transformation relationship between the lidar coordinate system and the terminal coordinate system, thereby calibrating the mapping relationship between the lidar coordinate system and the terminal coordinate system.

[0054] In one specific embodiment, the mapping relationship between the lidar coordinate system and the end-point coordinate system is determined by using the first spatial positions of at least four preset markers in the lidar coordinate system as independent variables and the third spatial positions of at least four preset markers in the end-point coordinate system as dependent variables; this mapping relationship is then used as the calibration result. Specifically, the transformation matrix is ​​solved by using the Iterative Closest Point (ICP) search method / Newton's iteration method / Singular Value Decomposition (SVD) algorithm to obtain the mapping relationship between the lidar coordinate system and the end-point coordinate system, which is then used as the calibration result.

[0055] For example, the mapping relationship between the lidar coordinate system and the end-point coordinate system is determined by the singular value decomposition algorithm, with at least four preset markers. It is understood that if the transformation matrix is ​​solved using the SVD algorithm for the coordinates of the n (n≥4) preset markers in the lidar coordinate system and the corresponding n pairs of non-collinear matching points in the end-point coordinate system, a unique solution can be obtained. Specifically, firstly, the average coordinates of the point cloud dataset of each preset marker in the LiDAR coordinate system are calculated, and the average coordinates of each preset marker in the end-point coordinate system are also calculated. Secondly, the average coordinates in the LiDAR coordinate system are subtracted from the coordinates of the point cloud dataset of each preset marker in the LiDAR coordinate system to obtain the decentralized coordinates in the LiDAR coordinate system. The average coordinates in the base coordinate system are subtracted from the coordinates of each preset marker in the end-point coordinate system to obtain the decentralized coordinates in the end-point coordinate system. Then, the decentralized coordinates in the LiDAR coordinate system and the decentralized coordinates in the end-point coordinate system are multiplied to obtain the matrix about the first spatial position and the third spatial position. Finally, singular value decomposition is performed on this matrix to obtain the attitude matrix. Based on the attitude matrix, the rotation matrix between the LiDAR coordinate system and the end-point coordinate system is obtained, which is the mapping relationship between the LiDAR coordinate system and the end-point coordinate system. This mapping relationship is used as the calibration result between the LiDAR coordinate system and the base coordinate system.

[0056] The technical solution of this embodiment calibrates the mapping relationship between the lidar coordinate system and the end-effector coordinate system by obtaining the spatial position of the preset marker in the lidar coordinate system and the end-effector coordinate system, thereby obtaining calibration results for surgical navigation. This improves the accuracy of lidar for surgical navigation and reduces the cost of the surgical navigation system.

[0057] Figure 3This is a flowchart of another coordinate transformation calibration method provided by an embodiment of the present invention. This embodiment belongs to the same inventive concept as the coordinate transformation calibration method in the above embodiment. Based on the above embodiment, after calibrating the mapping relationship between the lidar coordinate system and the end-effector coordinate system, the following steps are added: controlling the end-effector of the robotic arm to move to a second target position, the second target position being located above the target area of ​​the target object, and the distance between the second target position and the body surface directly above the target area is a set calibration distance; acquiring a lidar image of the target object; performing image fusion between the lidar image and a stored medical image to obtain a fused image, the medical image including a planned target path; controlling the end-effector of the robotic arm to drive the surgical instruments to move according to the correspondence and mapping relationship between the target path in the fused image and the lidar image.

[0058] S310. Adjust the end effector of the robotic arm to the first target position so that the distance between the end effector of the robotic arm and the calibration plate is the set calibration distance. At least four preset markers are set on the calibration plate. The calibration plate is rigidly connected to the base of the robotic arm. The height of the at least four preset markers corresponds to the height of at least four key points on the body surface of the corresponding human body area.

[0059] S320. Obtain the first spatial positions of at least four preset markers in the lidar coordinate system and the second spatial positions of at least four preset markers in the base coordinate system corresponding to the robotic arm, wherein the lidar is set at the end of the robotic arm and the mapping relationship between the base coordinate system and the end coordinate system of the robotic arm is known.

[0060] S330. The mapping relationship between the lidar coordinate system and the terminal coordinate system is calibrated, with the first spatial position of at least four preset markers in the lidar coordinate system as independent variables and the third spatial position of at least four preset markers in the terminal coordinate system as dependent variables.

[0061] S3401. Control the end effector of the robotic arm to move to the second target position. The second target position is located above the target area of ​​the target object, and the distance between the second target position and the surface of the target area is the set calibration distance.

[0062] Since the mapping relationship between the lidar coordinate system and the end effector coordinate system at the set calibration distance has been calibrated, in order to achieve further registration between the coordinate system of the medical image and the coordinate system of the lidar image, the end effector of the robotic arm is controlled to move to the second target position. The second target position is located above the target area of ​​the target object, and the distance between the second target position and the body surface directly above the target area is the set calibration distance.

[0063] S3402. Acquire the lidar image of the target object.

[0064] Specifically, when the robotic arm's end effector is at the second target position, it acquires a LiDAR image of the target object captured by the LiDAR. It can be understood that the acquired LiDAR image includes the target area of ​​the target object.

[0065] S3403. Perform image fusion between the lidar image and the stored medical image to obtain a fused image, wherein the medical image includes the planned target path.

[0066] Medical images can be clinical medical images of the target object and target area obtained preoperatively. The image type can be medical images obtained through computed tomography (CT), positron emission tomography (PET), digital radiography (DR), or magnetic resonance imaging (MRI), etc.

[0067] The target path is the surgical path annotated in the medical image before surgery. For example, it can be manually annotated by the doctor, or the surgical path of the target object can be generated by a trained neural network model and annotated in the medical image.

[0068] Using existing algorithms, programs, or tools, image fusion is performed between LiDAR images and medical images, including those with planned target paths, to achieve registration between the medical image coordinate system and the LiDAR image coordinate system.

[0069] In one specific embodiment, the medical image is a CT image of the target area of ​​the target object that has been planned for the target path. The CT image is fused with the LiDAR image of the target area that includes the target object to obtain a fused image, thereby achieving registration between the medical image coordinate system and the LiDAR image coordinate system.

[0070] Furthermore, image fusion is performed between the lidar image and the medical image to obtain a fused image, including:

[0071] Step b1: Select at least four corresponding feature points from the lidar image and the medical image.

[0072] Understandably, if the transformation matrix of the n (n≥4) feature points in the lidar coordinate system and the corresponding n pairs of non-collinear matching points corresponding to the coordinates of the feature points in the medical image coordinate system is solved using the SVD algorithm, a unique solution can be obtained. Therefore, at least four feature points are required.

[0073] Specifically, the coordinates of at least four feature points in the lidar image are determined as the fourth spatial location, and the coordinates corresponding to each feature point in the lidar image are determined in the medical image as the fifth spatial location.

[0074] Step b2: Based on the singular value decomposition algorithm, perform image fusion on the lidar image and the medical image according to the positions of at least four corresponding feature points in the lidar image and the medical image.

[0075] Specifically, the fourth spatial position of at least four feature points in the lidar image coordinate system and the fifth spatial position of the corresponding feature points in the medical image are solved by the singular value decomposition algorithm to realize the correspondence between the lidar image coordinate system and the medical image coordinate system; based on the correspondence between the lidar image coordinate system and the medical image coordinate system, the lidar image and the medical image are fused to obtain a fused image including the planned target path.

[0076] S3404. Based on the correspondence and mapping relationship between the target path in the fused image and the lidar image, control the end effector of the robotic arm to drive the surgical instruments to move.

[0077] First, the target path in the fused image is determined; second, based on the position of the target path in the LiDAR image, the target path in the LiDAR coordinate system is determined; then, based on the mapping relationship between the LiDAR coordinate system and the end effector coordinate system, the position of the target path in the end effector coordinate system is determined; finally, the end effector of the robotic arm is controlled to drive the surgical machinery to move according to the target path for surgical navigation.

[0078] Understandably, after obtaining the fused image, the LiDAR can be turned off, and the robotic arm can be controlled to move based on the planned target path in the fused image and the mapping relationship between the LiDAR coordinate system and the end-effector coordinate system for surgical navigation.

[0079] The technical solution of this embodiment, after calibrating the mapping relationship between the lidar coordinate system and the end-effector coordinate system, controls the movement of the robotic arm end-effector for surgical navigation based on the position of the surgical path in the end-effector coordinate system in the fused image of the lidar image and the medical image, further improving the accuracy and precision of lidar for surgical navigation.

[0080] Figure 4A This is a structural block diagram of a coordinate transformation calibration device provided according to an embodiment of the present invention. Figure 4A As shown, the device includes:

[0081] The robotic arm control module 401 is used to adjust the end of the robotic arm to the first target position so that the distance between the end of the robotic arm and the calibration plate is a set calibration distance. At least four preset markers are set on the calibration plate. The calibration plate is rigidly connected to the base of the robotic arm. The height of the at least four preset markers corresponds to the height of at least four key points on the body surface of the corresponding human body area.

[0082] The spatial position acquisition module 402 is used to acquire the first spatial position of at least four preset markers in the lidar coordinate system and the second spatial position of at least four preset markers in the base coordinate system corresponding to the robotic arm, wherein the lidar is set at the end of the robotic arm and the mapping relationship between the base coordinate system and the end coordinate system of the robotic arm is known.

[0083] The mapping relationship calibration module 403 is used to calibrate the mapping relationship between the lidar coordinate system and the end coordinate system, with the first spatial position of at least four preset markers in the lidar coordinate system as independent variables and the third spatial position of at least four preset markers in the end coordinate system as dependent variables.

[0084] Optionally, the spatial location acquisition module 402 is specifically used for:

[0085] Obtain the positional relationship of at least four preset markers relative to the base and the position of the preset known position points in the base coordinate system;

[0086] Based on the known positions and relationships of the known points in the base coordinate system and the known mapping relationship between the end-effector coordinate system and the base coordinate system corresponding to the current pose of the robotic arm, determine the third spatial positions of at least four preset markers in the end-effector coordinate system.

[0087] Optionally, the mapping relationship calibration module 403 is also used for:

[0088] The mapping relationship between the lidar coordinate system and the terminal coordinate system is determined by taking the first spatial position of at least four preset markers in the lidar coordinate system as independent variables and the third spatial position of at least four preset markers in the terminal coordinate system as dependent variables.

[0089] The mapping relationship is used as the calibration result.

[0090] Optional, such as Figure 4B As shown, the device also includes a surgical navigation module 404, which is used for:

[0091] The robotic arm end effector is controlled to move to the second target position, which is located above the target area of ​​the target object, and the distance between the second target position and the surface of the target area is the set calibration distance.

[0092] Acquire LiDAR images of the target object;

[0093] The lidar image is fused with a stored medical image to obtain a fused image, wherein the medical image includes a planned target path;

[0094] The robotic arm's end effector is controlled to move surgical instruments based on the correspondence and mapping relationship between the target path in the fused image and the LiDAR image.

[0095] Optionally, the surgical navigation module 404 is also used for:

[0096] Select at least four corresponding feature points from the lidar image and the medical image;

[0097] Based on the singular value decomposition algorithm, image fusion is performed on the lidar image and the medical image according to the positions of at least four corresponding feature points in the lidar image and the medical image.

[0098] The technical solution of this embodiment, through the cooperation of various modules, obtains the spatial position of the target area in the lidar coordinate system and the end-effector coordinate system, calibrates the mapping relationship between the lidar coordinate system and the end-effector coordinate system, improves the accuracy of lidar for surgical navigation, and reduces the cost of the surgical navigation system.

[0099] The coordinate transformation relationship calibration device provided in the embodiments of the present invention can execute the coordinate transformation relationship calibration method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0100] In some embodiments, the method for calibrating coordinate transformation relationships can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as... Figure 1 Storage unit 18 in the middle. In some embodiments, such as Figure 1 Part or all of the computer program shown may be loaded into and / or installed onto the surgical navigation system via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the calibration method for coordinate transformation relationships described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the calibration method for coordinate transformation relationships by any other suitable means (e.g., by means of firmware).

[0101] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0102] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0103] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0104] To provide user interaction, the systems and techniques described herein can be implemented on a surgical navigation system having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the surgical navigation system. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0105] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet. The computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having client-server relationships with each other. Servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are hosting products within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0106] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0107] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for calibrating coordinate transformation relationships, characterized in that, The method includes: The end effector of the robotic arm is adjusted to the first target position so that the distance between the end effector of the robotic arm and the calibration plate is the set calibration distance. At least four preset markers are set on the calibration plate. The calibration plate is rigidly connected to the base of the robotic arm. The height of the at least four preset markers corresponds to the height of at least four key points on the body surface of the corresponding human body area. The preset markers are cone-shaped markers. The first spatial positions of at least four preset markers in the lidar coordinate system and the second spatial positions of the at least four preset markers in the base coordinate system corresponding to the robotic arm are obtained, wherein the lidar is set at the end of the robotic arm, and the mapping relationship between the base coordinate system and the end coordinate system of the robotic arm is known; Using the first spatial position of the at least four preset markers in the lidar coordinate system as independent variables and the third spatial position of the at least four preset markers in the end coordinate system as dependent variables, the mapping relationship between the lidar coordinate system and the end coordinate system is calibrated. The step of calibrating the mapping relationship between the lidar coordinate system and the end-point coordinate system, using the first spatial positions of the at least four preset markers in the lidar coordinate system as independent variables and the third spatial positions of the at least four preset markers in the end-point coordinate system as dependent variables, includes: Calculate the average coordinates of the point cloud dataset of each preset marker in the lidar coordinate system, and calculate the average coordinates of each preset marker in the end coordinate system; Subtract the average coordinates in the lidar coordinate system from the coordinates of the point cloud dataset of each preset marker in the lidar coordinate system to obtain the decentralized coordinates in the lidar coordinate system. Subtract the average coordinates in the terminal coordinate system from the coordinates of each preset marker in the terminal coordinate system to obtain the decentralized coordinates in the terminal coordinate system. By multiplying the decentralized coordinates in the lidar coordinate system with the decentralized coordinates in the terminal coordinate system, a matrix is ​​obtained regarding the first spatial position and the third spatial position. The singular value decomposition of the matrix is ​​used to obtain the attitude matrix. Based on the attitude matrix, the rotation matrix between the lidar coordinate system and the end-effector coordinate system is obtained, which is the mapping relationship between the lidar coordinate system and the end-effector coordinate system. The mapping relationship is used as the calibration result between the lidar coordinate system and the base coordinate system.

2. The method according to claim 1, characterized in that, The following steps are used to obtain the third spatial position of the at least four preset markers in the end-effector coordinate system corresponding to the robotic arm: Obtain the positional relationship between the at least four preset markers and the known position points relative to the base, as well as the positions of the known position points in the base coordinate system; Based on the positions of the known points in the base coordinate system, the positional relationships, and the known mapping relationship between the end-effector coordinate system and the base coordinate system corresponding to the current pose of the robotic arm, the third spatial positions of the at least four preset markers in the end-effector coordinate system are determined.

3. A calibration device for coordinate transformation relationships, characterized in that, include: A robotic arm control module is used to adjust the end effector of the robotic arm to a first target position so that the distance between the end effector of the robotic arm and the calibration plate is a set calibration distance. The calibration plate is provided with at least four preset markers. The calibration plate is rigidly connected to the base of the robotic arm. The height of the at least four preset markers corresponds to the height of at least four key points on the body surface of the corresponding human body area. The preset markers are conical markers. A spatial position acquisition module is used to acquire the first spatial positions of at least four preset markers in the lidar coordinate system and the second spatial positions of the at least four preset markers in the base coordinate system corresponding to the robotic arm, wherein the lidar is set at the end of the robotic arm, and the mapping relationship between the base coordinate system and the end coordinate system of the robotic arm is known; The mapping relationship calibration module is used to calibrate the mapping relationship between the lidar coordinate system and the end coordinate system, with the first spatial position of the at least four preset markers in the lidar coordinate system as the independent variable and the third spatial position of the at least four preset markers in the end coordinate system as the dependent variable. The mapping relationship calibration module is specifically used for: Calculate the average coordinates of the point cloud dataset of each preset marker in the lidar coordinate system, and calculate the average coordinates of each preset marker in the end coordinate system; Subtract the average coordinates in the lidar coordinate system from the coordinates of the point cloud dataset of each preset marker in the lidar coordinate system to obtain the decentralized coordinates in the lidar coordinate system. Subtract the average coordinates in the terminal coordinate system from the coordinates of each preset marker in the terminal coordinate system to obtain the decentralized coordinates in the terminal coordinate system. By multiplying the decentralized coordinates in the lidar coordinate system with the decentralized coordinates in the terminal coordinate system, a matrix is ​​obtained regarding the first spatial position and the third spatial position. The singular value decomposition of the matrix is ​​used to obtain the attitude matrix. Based on the attitude matrix, the rotation matrix between the lidar coordinate system and the end-effector coordinate system is obtained, which is the mapping relationship between the lidar coordinate system and the end-effector coordinate system. The mapping relationship is used as the calibration result between the lidar coordinate system and the base coordinate system.

4. A surgical navigation system, characterized in that, The surgical navigation system includes: A robotic arm, the end of which is used to hold surgical instruments; The calibration plate is configured with at least four preset markers and is rigidly connected to the base of the robotic arm. The height of the at least four preset markers corresponds to the height of at least four key points on the body surface of the corresponding human body region. A lidar device is disposed at the end of the robotic arm to obtain the first spatial positions of the at least four preset markers in the lidar coordinate system. At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to obtain the second spatial position of the at least four preset markers in the base coordinate system of the robotic arm, and to perform the calibration method of the coordinate transformation relationship according to any one of claims 1-2.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the calibration method for the coordinate transformation relationship as described in any one of claims 1-2.

Images