Registration method and device, equipment and storage medium

Abstract

The invention relates to the technical field of intraoperative navigation, provides a registration method, device and equipment and a storage medium, and can improve the automation and intelligence of a registration process. The method comprises the following steps: reconstructing an execution space comprising imaging equipment and a target object to obtain a reconstructed space; according to the reconstruction space, obtaining a relative pose of the imaging device and the target object in the execution space; based on the poses of the target object in the planning space and the execution space, the relative poses of the imaging device and the target object in the execution space are converted into the planning space, and the relative poses of the imaging device and the target object in the planning space are obtained; and obtaining a registration matrix based on the relative pose of the imaging device and the target object in the planning space.

Description

Technical Field

[0001] The present application relates to the technical field of intraoperative navigation, and particularly to a registration method, device, computer device, storage medium, and computer program product. Background Art

[0002] Referring to Figure 1 , in an intraoperative navigation scenario based on preoperative image planning, a target object can be scanned preoperatively to obtain a preoperative three-dimensional (3D) image, and a surgical path can be planned on the preoperative 3D image to obtain a planned path; wherein, the space corresponding to the surgical path planning can be referred to as a planning space.

[0003] During the surgical procedure, a robotic arm can be navigated based on the planned path, and the corresponding operating space during the operation can be referred to as an execution space. When navigating the robotic arm based on the planned path, it is necessary to solve the transformation relationship between the planning space and the execution space in advance, so as to map the planned path from the planning space to the execution space. The process of solving the transformation relationship between the planning space and the execution space is generally referred to as registration.

[0004] In some intraoperative scenarios, an intraoperative two-dimensional (2D) image of the target object is obtained by using a radiographic machine such as a C-arm, so the above-mentioned registration is also referred to as 2D-3D registration.

[0005] The general idea of 2D-3D registration is as follows: in the planning space, the preoperative 3D image of the target object is projected based on a registration matrix (the registration matrix can be referred to as a projection pose) to obtain a preoperative 2D image, and based on the difference between the preoperative 2D image and the intraoperative 2D image, the registration matrix is optimized. When the difference between the preoperative 2D image and the intraoperative 2D image converges to an acceptable range, the iteration of the registration matrix is stopped.

[0006] Among them, the initial registration matrix required for iteration is generally given through manual interaction, which makes the registration process cumbersome. Summary of the Invention

[0007] Based on this, it is necessary to provide a registration method, device, computer device, storage medium, and computer program product for the above technical problems.

[0008] The present application provides a registration method, and the method includes:

[0009] Reconstruct an execution space including an imaging device and a target object to obtain a reconstructed space;

[0010] According to the reconstructed space, obtain the relative pose of the imaging device and the target object in the execution space;

[0011] Based on the poses of the target object in the planning space and the execution space, convert the relative pose between the imaging device and the target object in the execution space to the planning space, and obtain the relative pose between the imaging device and the target object in the planning space;

[0012] Based on the relative pose between the imaging device and the target object in the planning space, obtain a registration matrix.

[0013] In one embodiment, the method further includes:

[0014] Obtain a first three-dimensional model of the target object in the reconstruction space;

[0015] Based on the reconstruction pose of the first three-dimensional model in the reconstruction space, obtain a first pose of the target object in the execution space.

[0016] In one embodiment, based on the reconstruction pose of the first three-dimensional model in the reconstruction space, obtaining the first pose of the target object in the execution space includes:

[0017] The target object includes a target part. Perform pose analysis on the model part corresponding to the target part on the first three-dimensional model to obtain the reconstruction pose of the model part in the reconstruction space;

[0018] According to the reconstruction pose of the model part in the reconstruction space, obtain the first pose of the target part in the execution space.

[0019] In one embodiment, performing pose analysis on the model part corresponding to the target part on the first three-dimensional model to obtain the reconstruction pose of the model part in the reconstruction space includes:

[0020] Input the surface three-dimensional model included in the first three-dimensional model into a pre-constructed part pose estimation network;

[0021] According to the result output by the part pose estimation network, obtain the reconstruction pose of the model part in the reconstruction space.

[0022] In one embodiment, the method further includes:

[0023] Obtain a second three-dimensional model of the imaging device in the reconstruction space;

[0024] Based on the reconstruction pose of the second three-dimensional model in the reconstruction space, obtain a second pose of the imaging device in the execution space.

[0025] In one embodiment, according to the reconstruction space, obtaining the relative pose between the imaging device and the target object in the execution space includes:

[0026] Based on the reconstructed space, obtain the relative pose in the reconstructed space between the second three-dimensional model of the imaging device and the first three-dimensional model of the target object;

[0027] According to the coordinate system conversion relationship between the reconstructed space and the execution space, convert the relative pose in the reconstructed space between the second three-dimensional model and the first three-dimensional model to the execution space, so as to obtain the relative pose in the execution space between the imaging device and the target object.

[0028] In one embodiment, a vision system is used to acquire three-dimensional data of the imaging device and the target object and perform reconstruction. The method further includes:

[0029] Obtain the pose of the vision system in the execution space;

[0030] Based on the poses of the vision system in the reconstructed space and the execution space, obtain the coordinate system conversion relationship between the reconstructed space and the execution space.

[0031] This application provides a registration device, and the device includes:

[0032] A reconstruction module, configured to reconstruct the execution space including the imaging device and the target object to obtain a reconstructed space;

[0033] A pose processing module, configured to obtain the relative pose in the execution space between the imaging device and the target object according to the reconstructed space;

[0034] The pose processing module is further configured to, based on the poses of the target object in the planning space and the execution space, convert the relative pose in the execution space between the imaging device and the target object to the planning space, so as to obtain the relative pose in the planning space between the imaging device and the target object;

[0035] A registration processing module, configured to obtain a registration matrix based on the relative pose in the planning space between the imaging device and the target object.

[0036] This application provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the above method.

[0037] This application provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to perform the above method.

[0038] This application provides a computer program product, on which a computer program is stored, and the computer program is executed by a processor to perform the above method.

[0039] This application reconstructs the execution space including the imaging device and the target object to obtain a reconstructed space. Based on this reconstructed space, the relative pose of the imaging device and the target object in the execution space can be obtained. Based on the poses of the target object in the planning space and the execution space, the relative pose of the imaging device and the target object in the execution space is transformed to the planning space to obtain the relative pose of the imaging device and the target object in the planning space. Based on the relative pose of the imaging device and the target object in the planning space, an initial registration matrix required for iteration is obtained. This initial registration matrix is more effective compared to a randomly set one. Iterating with this initial registration matrix can obtain a registration matrix with higher accuracy, thereby more accurately transforming the planned path from the planning space to the execution space, and there is no need for manual interaction to give the initial registration matrix, greatly reducing the manual interaction process and improving the automation and intelligence of the registration process. Description of the Drawings

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the following described drawings are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0041] Figure 1 It is an application environment diagram for intraoperative navigation based on a planned path in an embodiment;

[0042] Figure 2 It is a schematic flowchart of a registration method in an embodiment;

[0043] Figure 3(a) is a schematic diagram of a SMPL human model with surface parameterization in an embodiment;

[0044] Figure 3(b) is a schematic flowchart of the reconstruction process of a three-dimensional model of the human body surface in an embodiment;

[0045] Figure 3(c) is a schematic diagram of bone prediction based on a three-dimensional surface model in an embodiment;

[0046] Figure 3(d) is a schematic diagram from the real execution space to the virtual reconstructed space in an embodiment;

[0047] Figure 3(e) is an application scenario diagram of a registration method in an embodiment;

[0048] Figure 3(f) is a schematic diagram of the execution space in an embodiment;

[0049] Figure 4 It is a structural block diagram of a registration device in an embodiment;

[0050] Figure 5Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0051] To make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] Referring to "embodiments" in the present application means that the specific features, structures or characteristics described in connection with the embodiments may be included in at least one embodiment of the present application. The phrase appears in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described in the present application may be combined with other embodiments.

[0053] The method provided by the present application can automatically give a relatively effective initial registration matrix without manual interaction, which can improve the automation and intelligence of 2D-3D registration. This method can be executed by a computer device and includes Figure 2 The steps shown:

[0054] Step S201: Reconstruct the execution space including the imaging device and the target object to obtain a reconstructed space.

[0055] The imaging device can perform projection imaging on the target object during the operation to obtain intraoperative two-dimensional images. The target object includes, but is not limited to, humans or other animals.

[0056] The execution space belongs to the real space, and the execution space includes the imaging device and the target object. The present application can use vision technology to reconstruct the execution space to obtain a reconstructed space, which belongs to the virtual space. Therefore, the reconstructed space includes the three-dimensional model of the imaging device and the three-dimensional model of the target object. For the sake of distinction, the three-dimensional model of the target object is called the first three-dimensional model, and the three-dimensional model of the imaging device is called the second three-dimensional model.

[0057] Step S202: Obtain the relative pose of the imaging device and the target object in the execution space according to the reconstructed space.

[0058] The reconstructed space obtained by reconstructing the execution space can be understood as a digital reflection of the execution space. Therefore, after the reconstructed space is obtained, the relative pose of the second three-dimensional model and the first three-dimensional model in the reconstructed space can be obtained. According to the relative pose of the second three-dimensional model and the first three-dimensional model in the reconstructed space, the relative pose of the imaging device and the target object in the execution space can be obtained.

[0059] This application relates to the description of the pose of an object in a certain space. It can be understood that the specific representation of the pose of an object in a certain space can be: the pose of the object in the space coordinate system; this application also relates to the description of the relative pose of two objects in a certain space. It can be understood that the specific representation of the relative pose of two objects in a certain space can be: the relative pose of the two objects in the space coordinate system.

[0060] Step S203: Based on the poses of the target object in the planning space and the execution space, convert the relative pose between the imaging device and the target object in the execution space to the planning space, and obtain the relative pose between the imaging device and the target object in the planning space.

[0061] During preoperative path planning, the preoperative three-dimensional image of the target object has a certain pose in the planning space, and this pose can be recorded. Subsequently, during registration, the pose of the preoperative three-dimensional image of the target object in the planning space can be obtained.

[0062] After the reconstruction space is obtained, the pose of the first three-dimensional model in the reconstruction space can be obtained; since the reconstruction space is a digital reflection of the execution space, therefore, based on the pose of the first three-dimensional model in the reconstruction space, the pose of the target object in the execution space can be obtained.

[0063] Based on the pose of the preoperative three-dimensional image of the target object in the planning space and the pose of the target object in the execution space, the coordinate transformation relationship between the planning space and the execution space can be obtained. According to the coordinate transformation relationship between the planning space and the execution space, the relative pose between the imaging device and the target object in the execution space can be transferred to the planning space, thereby obtaining the relative pose between the imaging device and the target object in the planning space.

[0064] Step S204: Based on the relative pose between the imaging device and the target object in the planning space, obtain the registration matrix.

[0065] After obtaining the relative pose between the imaging device and the target object in the planning space, the initial registration matrix required for iteration can be determined according to this relative pose. Using this initial registration matrix for iterative optimization, the registration matrix can be obtained. Based on this registration matrix, the planned path can be projected in the planning space to obtain the projected path, and intraoperative navigation can be performed based on the projected path.

[0066] According to the relative pose between the imaging device and the target object in the planning space, the initial registration matrix required for iteration is obtained. Specifically, according to the relative pose between the imaging device and the target object in the planning space, the projection geometric relationship between the imaging device and the target object in the execution space can be obtained, and the initial registration matrix is obtained according to this projection geometric relationship.

[0067] In some scenarios, the registration matrix obtained by iteratively optimizing the initial registration matrix can be used as the rough registration matrix, and the rough registration matrix can be further adjusted to obtain a registration matrix with higher accuracy.

[0068] In the above registration method, the execution space including the imaging device and the target object is reconstructed to obtain a reconstructed space. Based on the reconstructed space, the relative pose of the imaging device and the target object in the execution space can be obtained; based on the poses of the target object in the planning space and the execution space, the relative pose of the imaging device and the target object in the execution space is transformed to the planning space to obtain the relative pose of the imaging device and the target object in the planning space; based on the relative pose of the imaging device and the target object in the planning space, the initial registration matrix required for iteration is obtained. This initial registration matrix is more effective compared to being randomly set. Iterating with this initial registration matrix can obtain a registration matrix with higher accuracy, thereby more accurately transforming the planned path from the planning space to the execution space, and there is no need for manual interaction to give the initial registration matrix, greatly reducing the manual interaction process and improving the automation and intelligence of the registration process.

[0069] In one embodiment, the method provided by this application further includes: obtaining a first three-dimensional model of the target object in the reconstructed space; and obtaining a first pose of the target object in the execution space based on the reconstructed pose of the first three-dimensional model in the reconstructed space.

[0070] After the reconstructed space is obtained, a first three-dimensional model of the target object in the reconstructed space can be obtained. The pose of the first three-dimensional model in the reconstructed space can be referred to as the reconstructed pose of the first three-dimensional model in the reconstructed space. Since the reconstructed space is a digital reflection of the execution space, therefore, according to the reconstructed pose of the first three-dimensional model in the reconstructed space, the pose of the target object in the execution space can be obtained. To distinguish it from the pose of the imaging device in the execution space, the pose of the target object in the execution space is referred to as the first pose of the target object in the execution space, and the pose of the imaging device in the execution space is referred to as the second pose of the imaging device in the execution space.

[0071] In one embodiment, obtaining the first pose of the target object in the execution space based on the reconstructed pose of the first three-dimensional model in the reconstructed space may specifically include: the target object includes a target part, performing pose analysis on the model part corresponding to the target part on the first three-dimensional model to obtain the reconstructed pose of the model part in the reconstructed space; and obtaining the first pose of the target part in the execution space according to the reconstructed pose of the model part in the reconstructed space.

[0072] Any part of the target object can be used as the target part. In the case where the target object is a human body, parts such as limbs, head, torso, and bones can be used as the target part.

[0073] Exemplarily, if the bone is used as the target part, the model part can be called the model bone. After obtaining the first three-dimensional model in this embodiment, the pose analysis can be performed on the model bone corresponding to the target object's bone on the first three-dimensional model to obtain the pose of the model bone in the reconstruction space, and the pose of the model bone in the reconstruction space is called the reconstruction pose of the model bone in the reconstruction space. According to the reconstruction pose of the model bone in the reconstruction space, the pose of the target object's bone in the execution space can be obtained, and the pose of the target object's bone in the execution space can represent the first pose of the target object in the execution space. Therefore, the pose of the target object's bone in the execution space can be called the first pose of the target object's bone in the execution space.

[0074] In one embodiment, performing pose analysis on the model part corresponding to the target part on the first three-dimensional model to obtain the reconstruction pose of the model part in the reconstruction space includes: inputting the surface three-dimensional model included in the first three-dimensional model into a pre-constructed part pose estimation network; and obtaining the reconstruction pose of the model part in the reconstruction space according to the result output by the part pose estimation network.

[0075] After obtaining the first three-dimensional model, the mesh surface model (Mesh model) of the first three-dimensional model can be reconstructed, and the mesh surface model is used as the surface three-dimensional model of the first three-dimensional model, and the surface three-dimensional model is input into a pre-constructed part pose estimation network. If the target part is a bone, then the reconstruction pose of the model bone in the reconstruction space can be obtained according to the result output by the part pose estimation network.

[0076] In some specific scenarios, when the target object is a human body, the SMPLify method can be used to capture the human body pose through vision technology, collect the human body two-dimensional image, and reconstruct the mesh surface model of the human body based on the human body two-dimensional image. SMPLify is a human motion capture method based on the SMPL (Skinned Multi-Person Linear model) human model. Its main training steps are as follows:

[0077] a) SMPL human model: The SMPL human model used by the SMPLify method can include the human body shape and motion changes learned from large-scale three-dimensional human scans. The SMPL human model can be controlled by three groups of parameters: body shape parameters, pose parameters, and deformation parameters.

[0078] b) 2D human key point detection: A 2D human key point detector (such as OpenPose) can be used to perform 2D human key point detection on the human body two-dimensional image as a two-dimensional image to obtain the input of the SMPL human model.

[0079] c) Initialization of SMPL parameters: Using a simple projection model, project the detected 2D human key points onto the SMPL human model to obtain the initial values of the three groups of SMPL parameters.

[0080] d) Construction of data items: Construct data items containing the reprojection error between the detected 2D human key points and the SMPL human model. This item is used to evaluate the matching degree under the current SMPL parameters.

[0081] e) Optimization and solution: Use optimization methods such as gradient descent to adjust the three groups of SMPL parameters, minimize the total loss function, and obtain the optimal parameters.

[0082] Referring to FIGS. 3(a) and 3(b), FIG. 3(a) is a surface-parameterized SMPL human model, and FIG. 3(b) is a schematic diagram of reconstructing a mesh surface model based on a human two-dimensional image.

[0083] In actual use, after obtaining a human two-dimensional image, based on the human two-dimensional image and the SMPL optimal parameters, a better SMPL human model is obtained, and thus a mesh surface model is obtained. After obtaining the mesh surface model of the human body, bone pose prediction can be performed based on the mesh surface model of the human body to obtain the pose of the model bones in the reconstruction space, as shown in FIG. 3(c).

[0084] In one embodiment, the method provided by this application further includes: obtaining a second three-dimensional model of the imaging device in the reconstruction space; obtaining a second pose of the imaging device in the execution space based on the reconstruction pose of the second three-dimensional model in the reconstruction space.

[0085] After reconstructing the reconstruction space, a second three-dimensional model of the imaging device in the reconstruction space can be obtained, and the pose of the second three-dimensional model in the reconstruction space can be referred to as the reconstruction pose of the second three-dimensional model in the reconstruction space. Since the reconstruction space is a digital reflection of the execution space, according to the reconstruction pose of the second three-dimensional model in the reconstruction space, the second pose of the imaging device in the execution space can be obtained.

[0086] In one embodiment, obtaining the relative pose between the imaging device and the target object in the execution space according to the reconstruction space includes: obtaining the relative pose between the second three-dimensional model of the imaging device and the first three-dimensional model of the target object in the reconstruction space according to the reconstruction space; according to the coordinate system conversion relationship between the reconstruction space and the execution space, converting the relative pose between the second three-dimensional model and the first three-dimensional model in the reconstruction space to the execution space to obtain the relative pose between the imaging device and the target object in the execution space.

[0087] After obtaining the reconstructed space through reconstruction, the relative pose of the second three-dimensional model of the imaging device and the first three-dimensional model of the target object in the reconstructed space can be obtained. According to the coordinate system conversion relationship between the reconstructed space and the execution space, the relative pose of the second three-dimensional model and the first three-dimensional model in the reconstructed space can be converted into the execution space, and the relative pose of the second three-dimensional model and the first three-dimensional model in the execution space can be obtained. Since the reconstructed space is a digital reflection of the execution space, the relative pose of the second three-dimensional model and the first three-dimensional model in the execution space can be used as the relative pose of the imaging device and the target object in the execution space.

[0088] In one embodiment, a vision system is used to acquire the three-dimensional data of the imaging device and the target object and perform reconstruction. The method provided in this application further includes: obtaining the pose of the vision system in the execution space; based on the poses of the vision system in the reconstructed space and the execution space, obtaining the coordinate system conversion relationship between the reconstructed space and the execution space.

[0089] In this embodiment, a vision system can be used to acquire the three-dimensional data of the imaging device and the three-dimensional data of the target object. Compared with the vision systems of humans or other animals, the vision system used in this application belongs to a machine vision system. Based on the three-dimensional data of the imaging device and the three-dimensional data of the target object, the execution space including the imaging device and the target object is reconstructed to obtain the reconstructed space.

[0090] Specifically, the visible light camera of the vision system can be used to acquire the 2D image data of the imaging device and the target object respectively, and the three-dimensional data of the imaging device and the target object can be obtained through algorithms; or the 3D depth camera of the vision system can be directly used to collect the surface point cloud of the imaging device or the target object respectively, and the surface point cloud belongs to the three-dimensional data. In some embodiments, the reconstruction process can be completed jointly by the point cloud and the engineering design model through the advantages of the existing device engineering model.

[0091] When this application uses a vision system to collect the three-dimensional data of the imaging device and the target object and perform reconstruction, the pose of the vision system in the execution space can be obtained through calibration. After reconstructing the reconstructed space, the pose of the vision system in the reconstructed space can also be obtained, and based on the poses of the vision system in the reconstructed space and the execution space, the coordinate system conversion relationship between the reconstructed space and the execution space can be obtained.

[0092] To better understand the above method, the following details an application example of the registration method of this application. In this application example, the imaging device uses an X-ray machine with a C-arm (hereinafter referred to as the C-arm), the target object is a human body, and the target part is the bone.

[0093] In this application example, the execution spaces of the C-arm and the human body are reconstructed through the vision system to obtain the reconstructed space, as shown in Fig. 3(d).

[0094] Specifically, referring to Fig. 3(e), a vision system (such as a 3D depth camera, a structured light camera, etc.) is used to obtain the three-dimensional data of the C-arm (step S301). Based on the three-dimensional data of the C-arm, a mesh surface model of the C-arm is reconstructed (step S302).

[0095] Specifically, the vision system may include the 2 calibrated 3D depth cameras shown in Fig. 3(f). Among them, the number of 3D depth cameras is not limited to 2 and can be set according to actual needs. The surface point cloud of the C-arm is collected by the 3D depth cameras of the vision system, and a mesh surface model of the C-arm is reconstructed based on the surface point cloud. The mesh surface model of the C-arm is used as the second three-dimensional model, so as to obtain the pose of the second three-dimensional model in the reconstructed space.

[0096] Referring to Fig. 3(e) again, the vision system can be used to collect the 2D image data of the human body, and the three-dimensional data of the human body is obtained through an algorithm (step S303). Based on the three-dimensional data of the human body, a mesh surface model of the human body is reconstructed (step S304). Based on the mesh surface model of the human body, the bone pose is estimated to obtain the pose of the model bones in the reconstructed space (step S305).

[0097] By calibrating the vision system, the pose of the vision system in the execution space can be obtained. Based on the pose of the vision system in the reconstructed space and the pose of the vision system in the execution space, the coordinate system conversion relationship between the reconstructed space and the execution space can be obtained.

[0098] According to the coordinate system conversion relationship between the reconstructed space and the execution space and the pose of the second three-dimensional model in the reconstructed space, the pose of the C-arm in the execution space can be obtained. According to the coordinate system conversion relationship between the reconstructed space and the execution space and the pose of the model bones in the reconstructed space, the pose of the human bones in the execution space can be obtained.

[0099] During preoperative path planning, the preoperative three-dimensional image of the human body has a certain pose in the planning space, and this pose can be recorded. Subsequently, during registration, the pose of the preoperative three-dimensional image of the human body in the planning space can be obtained. According to the pose of the preoperative three-dimensional image of the human body in the planning space, the pose of the bones in the preoperative three-dimensional image in the planning space can be determined.

[0100] Based on the poses of the bones in the preoperative three-dimensional image in the planning space and the poses of the human bones in the execution space, the coordinate system transformation relationship between the planning space and the execution space is obtained. According to the coordinate system transformation relationship between the planning space and the execution space, the relative pose of the C-arm and the human body in the execution space can be transferred to the planning space, so as to obtain the relative pose of the C-arm and the human body in the planning space. According to the relative pose of the C-arm and the human body in the planning space, the projection geometric relationship between the C-arm and the human body in the execution space can be obtained, and the initial registration matrix can be obtained based on this projection geometric relationship. Iterative optimization is performed with this initial registration matrix to obtain the registration matrix. Based on this registration matrix, the planned path can be projected in the planning space to obtain the projected path, and intraoperative navigation can be performed based on the projected path.

[0101] This application example uses non-contact vision technology to reconstruct the execution space and infer the bone pose on the mesh surface model of the human body, avoiding physical intervention on devices such as patients and C-arms, as well as many cumbersome operations, and solving the problem that the tracking array is prone to falling off and shifting during the operation, resulting in errors. Moreover, this application example can give a relatively effective initial registration matrix without manual interaction, improving the automation and intelligence of the registration process.

[0102] The method provided by this application example has the following technical advantages:

[0103] (1) Non-invasive and fully automatic: As a non-contact and markerless technology, the method provided by this application example has no physical intervention on devices such as patients and C-arms, and realizes the fully automatic acquisition of the projection geometric relationship.

[0104] (2) High precision: 3D depth cameras and vision algorithms can achieve detailed three-dimensional reconstruction and bone pose prediction, providing a high-precision reference for obtaining the projection geometric relationship.

[0105] (3) Easy to integrate: It can be carried out with the help of common 3D depth cameras, and is easy to integrate with different surgical navigation systems to realize the acquisition and sharing of the projection geometric relationship.

[0106] (4) Low cost: Compared with the navigation system based on specific tracking devices, the method provided by this application example does not require additional complex navigation devices, and the overall cost is relatively low.

[0107] The method provided by this application example has great advantages in terms of non-invasiveness, precision, easy integration and low cost, and can be used in orthopedic navigation surgery guided by preoperative images.

[0108] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown in the direction of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0109] Based on the same inventive concept, an embodiment of the present application further provides a registration device for implementing the above-mentioned registration method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the registration device provided below can refer to the limitations on the registration method in the above text, and will not be repeated here.

[0110] In one embodiment, as Figure 4 shown, a registration device is provided, including:

[0111] A reconstruction module 401, configured to reconstruct an execution space including an imaging device and a target object to obtain a reconstructed space;

[0112] A pose processing module 402, configured to obtain the relative pose of the imaging device and the target object in the execution space according to the reconstructed space;

[0113] The pose processing module 402 is further configured to convert the relative pose of the imaging device and the target object in the execution space to the planning space based on the poses of the target object in the planning space and the execution space, to obtain the relative pose of the imaging device and the target object in the planning space;

[0114] A registration processing module 403, configured to obtain a registration matrix based on the relative pose of the imaging device and the target object in the planning space.

[0115] In one embodiment, the pose processing module 402 is further configured to: obtain a first three-dimensional model of the target object in the reconstructed space; and obtain a first pose of the target object in the execution space based on the reconstruction pose of the first three-dimensional model in the reconstructed space.

[0116] In one embodiment, the pose processing module 402 is further configured to: the target object includes a target part, perform pose analysis on the model part corresponding to the target part on the first three-dimensional model to obtain the reconstructed pose of the model part in the reconstruction space; and obtain the first pose of the target part in the execution space according to the reconstructed pose of the model part in the reconstruction space.

[0117] In one embodiment, the pose processing module 402 is further configured to: input the surface three-dimensional model included in the first three-dimensional model into a pre-constructed part pose estimation network; and obtain the reconstructed pose of the model part in the reconstruction space according to the result output by the part pose estimation network.

[0118] In one embodiment, the pose processing module 402 is further configured to: obtain a second three-dimensional model of the imaging device in the reconstruction space; and obtain the second pose of the imaging device in the execution space based on the reconstructed pose of the second three-dimensional model in the reconstruction space.

[0119] In one embodiment, the pose processing module 402 is further configured to: obtain the relative pose of the second three-dimensional model of the imaging device and the first three-dimensional model of the target object in the reconstruction space according to the reconstruction space; and convert the relative pose of the second three-dimensional model and the first three-dimensional model in the reconstruction space to the execution space according to the coordinate system conversion relationship between the reconstruction space and the execution space, so as to obtain the relative pose of the imaging device and the target object in the execution space.

[0120] In one embodiment, a vision system is used to acquire three-dimensional data of the imaging device and the target object and perform reconstruction. The device further includes a conversion relationship acquisition module, configured to: acquire the pose of the vision system in the execution space; and obtain the coordinate system conversion relationship between the reconstruction space and the execution space based on the poses of the vision system in the reconstruction space and the execution space.

[0121] Each module in the above registration device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0122] In an exemplary embodiment, a computer device is provided, and its internal structure diagram can be as Figure 5As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the data involved in the above method. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements a registration method.

[0123] Those skilled in the art can understand that Figure 5 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0124] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps in the above-mentioned method embodiments are implemented.

[0125] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0126] In one embodiment, a computer program product is provided, on which a computer program is stored. The computer program executes the steps in the above-mentioned method embodiments by the processor.

[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0128] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0129] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0130] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A registration method, characterized in that, The method includes: Reconstructing an execution space including an imaging device and a target object to obtain a reconstructed space; Obtaining a relative pose between the imaging device and the target object in the execution space according to the reconstructed space; Based on the poses of the target object in the planning space and the execution space, converting the relative pose between the imaging device and the target object in the execution space to the planning space to obtain the relative pose between the imaging device and the target object in the planning space; Obtaining a registration matrix based on the relative pose between the imaging device and the target object in the planning space.

2. The method according to claim 1, wherein The method further includes: Obtaining a first three-dimensional model of the target object in the reconstructed space; Obtaining a first pose of the target object in the execution space based on the reconstructed pose of the first three-dimensional model in the reconstructed space.

3. The method according to claim 2, wherein Obtaining a first pose of the target object in the execution space based on the reconstructed pose of the first three-dimensional model in the reconstructed space includes: The target object includes a target part. Performing pose analysis on a model part corresponding to the target part on the first three-dimensional model to obtain the reconstructed pose of the model part in the reconstructed space; Obtaining the first pose of the target part in the execution space according to the reconstructed pose of the model part in the reconstructed space.

4. The method according to claim 3, wherein Performing pose analysis on a model part corresponding to the target part on the first three-dimensional model to obtain the reconstructed pose of the model part in the reconstructed space includes: Inputting a surface three-dimensional model included in the first three-dimensional model into a pre-constructed part pose estimation network; Obtaining the reconstructed pose of the model part in the reconstructed space according to the result output by the part pose estimation network.

5. The method according to claim 1, characterized in that The method further includes: Obtaining a second three-dimensional model of the imaging device in the reconstructed space; Obtaining a second pose of the imaging device in the execution space based on the reconstructed pose of the second three-dimensional model in the reconstructed space.

6. The method according to claim 1, wherein According to the reconstructed space, obtaining a relative pose between the imaging device and the target object in the execution space includes: According to the reconstructed space, obtaining a relative pose between the second three-dimensional model of the imaging device and the first three-dimensional model of the target object in the reconstructed space; According to the coordinate system conversion relationship between the reconstructed space and the execution space, converting the relative pose between the second three-dimensional model and the first three-dimensional model in the reconstructed space to the execution space to obtain the relative pose between the imaging device and the target object in the execution space.

7. The method according to claim 6, characterized in that Using a vision system to acquire three-dimensional data of an imaging device and a target object and perform reconstruction. The method further includes: Obtaining the pose of the vision system in the execution space; Obtaining the coordinate system conversion relationship between the reconstructed space and the execution space based on the poses of the vision system in the reconstructed space and the execution space.

8. A registration device, characterized in that, The device includes: A reconstruction module for reconstructing an execution space including an imaging device and a target object to obtain a reconstructed space; A pose processing module for obtaining a relative pose between the imaging device and the target object in the execution space according to the reconstructed space; The pose processing module is further configured to convert the relative pose between the imaging device and the target object in the execution space to the planning space based on the poses of the target object in the planning space and the execution space, so as to obtain the relative pose between the imaging device and the target object in the planning space; The registration processing module is configured to obtain a registration matrix based on the relative pose between the imaging device and the target object in the planning space.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images