Method and device for aligning three-dimensional model pose and human pelvis pose

Abstract

The invention provides an alignment method and device for a three-dimensional model pose and a human pelvis pose, and relates to the technical field of medical image registration, and the method comprises the steps: obtaining pelvis 2D data of a patient in a pelvis minimally invasive surgery and a pre-constructed model pose of a three-dimensional model of the patient; the method comprises the following steps: determining a plurality of initial pose offset matrixes according to the positions of a plurality of interest points in pre-configured pelvis 3D data and the target position of each interest point in the body of a patient, determining a target pose offset matrix in the plurality of initial pose offset matrixes, and adjusting the model pose according to the target pose offset matrix to obtain a target pose. The model pose of the three-dimensional model is aligned with the pelvic pose of the patient. According to the method, the globally optimal solution is searched by adopting the step-by-step subdivision search strategy, the branching and trimming steps are realized, the scale of the search space is reduced, the algorithm efficiency is improved, and the method has higher adaptability and universality for the shape alignment problem.

Description

Technical Field

[0001] This application relates to the technical field of medical image registration, and more particularly, to a method and apparatus for aligning the pose of a three-dimensional model with the pose of a human pelvis. Background Art

[0002] Generally, the Procrustes analysis method is used to convert the 3D data of a patient collected by CT and the 2D data collected by X-ray. The rigid transformation matches the 3D interest points of the patient and calculates the pose deviation between the two to complete the registration. However, this method has certain limitations. First, the Procrustes analysis method depends on the one-to-one correspondence of point sets. In the presence of noise, partial point loss, or incorrect matching, the registration accuracy may be significantly affected. Second, this method is based on least squares optimization and is prone to falling into local optimal solutions, especially when the initial alignment is poor, it is difficult to guarantee a globally optimal registration result. In addition, the Procrustes analysis method does not fully utilize the geometric structure information between point sets, and its stability and robustness are limited in complex anatomical structures or high-noise environments. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a method and apparatus for aligning the pose of a three-dimensional model with the pose of a human pelvis to solve the above problems existing in the prior art and improve the registration accuracy and stability.

[0004] In a first aspect, a method for aligning the pose of a three-dimensional model with the pose of a human pelvis is provided, which may include:

[0005] Obtain the 2D data of the pelvis of a patient during minimally invasive pelvic surgery and the model pose of the three-dimensional model of the patient pre-constructed;

[0006] Determine a plurality of initial pose offset matrices through the interest point positions of a plurality of interest points in the pre-configured 3D data of the pelvis and the target positions of each interest point in the patient's body;

[0007] Divide the pose offset matrix queue composed of a plurality of initial pose offset matrices to obtain a plurality of initial spaces;

[0008] Recursively search in a plurality of initial spaces based on the configured initial transformation search range to obtain the initial space with the minimum alignment error; the minimum alignment error is calculated through a constructed alignment error function;

[0009] If the difference between the error upper bound and the error lower bound of the initial space with the minimum alignment error is less than the configured error threshold, determine the initial pose offset matrix corresponding to the initial space with the minimum alignment error as the target pose offset matrix;

[0010] Adjust the model pose according to the target pose offset matrix to obtain the target pose, which represents the current pelvic pose of the patient, so as to align the model pose of the three-dimensional model with the pelvic pose of the patient.

[0011] In a possible implementation, the construction of the alignment error function includes:

[0012] Based on the pelvic 2D data, the pelvic 3D data, and the interest point positions of multiple interest points in the pelvic 3D data, determine the initial positions of multiple interest points in the pelvic 2D data;

[0013] Based on the triangulation method, process the initial positions of each interest point to obtain the target positions of each interest point;

[0014] Based on the interest point positions of multiple interest points and the target positions of each interest point, construct an alignment error function.

[0015] In a possible implementation, the alignment error function is:

[0016]

[0017] where T* is the target pose offset matrix, T is the position pose offset matrix, X CT is a set of interest point positions of multiple interest points, is a set of target positions of each interest point in the patient's body.

[0018] In a possible implementation, the method further includes:

[0019] If the difference between the upper error bound and the lower error bound of the initial space of the minimum alignment error is not less than the configured error threshold, and the lower error bound is not greater than the minimum alignment error, then divide the initial space of the minimum error to obtain multiple subspaces;

[0020] In multiple subspaces, determine the subspace with the minimum alignment error;

[0021] If the difference between the upper error bound and the lower error bound of the subspace with the minimum alignment error is less than the configured error threshold, then determine the initial pose offset matrix corresponding to the subspace with the minimum alignment error as the target pose offset matrix;

[0022] If the difference between the upper error bound and the lower error bound of the subspace with the minimum alignment error is not less than the configured error threshold, and the lower error bound is not greater than the minimum alignment error, then divide the subspace with the minimum alignment error to obtain multiple new subspaces, and use the multiple new subspaces as the current subspaces, and return to execute the step: In multiple subspaces, determine the subspace with the minimum alignment error.

[0023] In a possible implementation, the method further includes:

[0024] If the difference between the upper error bound and the lower error bound of the subspace with the minimum alignment error is not less than the configured error threshold, and the lower error bound is greater than the minimum alignment error, then delete the subspace corresponding to the minimum alignment error from the multiple subspaces to obtain a new set of multiple subspaces, and return to execute the step: determine the subspace with the minimum error among the multiple subspaces.

[0025] In a possible implementation, the upper error bound is determined as follows:

[0026] Based on the first centroid of multiple points of interest in the pelvic 3D data, determine the first distance between the first centroid and the bounding box of the multiple points of interest in the pelvic 3D data;

[0027] Based on the second centroid of each point of interest in the patient's body, determine the second distance between the second centroid and the bounding box of each point of interest in the patient's body;

[0028] Determine the average value of the first distance and the second distance as the upper error bound.

[0029] In a possible implementation, the lower error bound is equal to the minimum alignment error.

[0030] In a second aspect, there is provided an alignment device for the pose of a three-dimensional model and the pose of a human pelvis, and the device may include:

[0031] An acquisition unit, configured to acquire the pelvic 2D data of a patient during a minimally invasive pelvic surgery and the model pose of a pre-constructed three-dimensional model of the patient;

[0032] A determination unit, configured to determine a plurality of initial pose offset matrices based on the positions of multiple points of interest in the pre-configured pelvic 3D data and the target positions of the respective points of interest in the patient's body;

[0033] A segmentation unit, configured to segment a pose offset matrix queue composed of a plurality of initial pose offset matrices to obtain a plurality of initial spaces;

[0034] A search unit, configured to recursively search in the plurality of initial spaces based on a configured initial transformation search range to obtain an initial space with the minimum alignment error; the minimum alignment error is calculated by a constructed alignment error function;

[0035] The determination unit is further configured to, if the difference between the upper error bound and the lower error bound of the initial space with the minimum alignment error is less than the configured error threshold, determine the initial pose offset matrix corresponding to the initial space with the minimum alignment error as the target pose offset matrix;

[0036] An adjustment unit for adjusting the model pose according to the target pose offset matrix to obtain a target pose, where the target pose represents the current pelvic pose of the patient, so as to align the model pose of the three-dimensional model with the pelvic pose of the patient.

[0037] In a third aspect, an electronic device is provided. The electronic device includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0038] The memory is used to store a computer program;

[0039] The processor is configured to implement the method steps described in any one of the first aspects when executing the program stored on the memory.

[0040] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and the computer program implements the method steps described in any one of the first aspects when executed by a processor.

[0041] This application provides a method for aligning the pose of a three-dimensional model with the pelvic pose of a human body. Compared with traditional shape alignment methods, this application has the following advantages: (1) Global optimal search: Avoiding the local optimal problem of the Procrustes method. (2) Robustness to mis-matched points: BnB effectively eliminates the influence of the alignment results of noise points through error bound constraints. (3) Efficient calculation: The pruning strategy reduces the search space and ensures high-precision alignment within a reasonable time. (4) Applicable to medical images: Optimized for 2D / 3D pelvic image registration to improve cross-modal alignment accuracy. Description of the Drawings

[0042] To more clearly illustrate the technical solutions of the embodiments of this application, the following will briefly introduce the drawings required to be used in the embodiments of this application. It should be understood that the following drawings only show some embodiments of this application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative efforts.

[0043] Figure 1 It is a system architecture diagram of a method for aligning the pose of a three-dimensional model with the pelvic pose of a human body provided by an embodiment of this application;

[0044] Figure 2 It is a flowchart of a method for aligning the pose of a three-dimensional model with the pelvic pose of a human body provided by an embodiment of this application;

[0045] Figure 3 It is a flowchart of transfer learning provided by an embodiment of this application;

[0046] Figure 4 A schematic structural diagram of the point-of-interest tracking network provided by an embodiment of the present application;

[0047] Figure 5 A schematic structural diagram of an alignment device for the pose of a three-dimensional model and the pose of a human pelvis provided by an embodiment of the present application;

[0048] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0049] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0050] A method for identifying the pose of a human pelvis provided by an embodiment of the application can be applied in Figure 1 the system architecture of the pelvic minimally invasive surgery navigation system shown in Figure 1 As shown, the system may include:

[0051] Image acquisition and processing module: This module is responsible for collecting the medical image data of the patient before and during the operation, such as preoperative CT (Computed Tomography), MRI (Magnetic Resonance Imaging), and intraoperative X-ray, etc., and is responsible for generating DRR images using the pelvic 3D data collected by CT for subsequent use. These image data are used to construct a three-dimensional model of the patient and perform registration operations to provide detailed anatomical information for the operation.

[0052] Surgical planning module: Based on the acquired image data, the doctor can perform virtual surgical plans through surgical planning software, including determining the best surgical path, simulating the position and size of the implant, etc. This helps to formulate the most optimized surgical plan before the actual operation.

[0053] Tracking and positioning module: The system uses an optical or electromagnetic tracker to monitor the position of the surgical instrument relative to the patient's body in real time. By matching the position information in the physical space with the pre-constructed three-dimensional model, it can provide accurate operation guidance for the surgeon. The present application is applied in this module to obtain the model pose of the pre-constructed three-dimensional model of the patient and the pelvic 2D data of the patient during pelvic minimally invasive surgery to execute a method for identifying the pose of a human pelvis provided by the present application.

[0054] Display and User Interface: An intuitive and easy-to-use user interface is crucial for operating such a system. It typically includes a touchscreen display, a head-mounted display, or other forms of visualization tools to facilitate the surgeon's viewing of detailed images of the surgical area and information on surgical planning.

[0055] Assistant Robots: It may also include one or more assistant robotic arms that can perform precise actions under the control of the doctor, such as cutting, drilling, or placing implants.

[0056] Safety and Quality Control Module: To ensure the safety and effectiveness of the surgery, the system also includes a series of safety measures, such as error detection, boundary alarms, and real-time feedback mechanisms, to help the doctor avoid potential operation mistakes.

[0057] During the tracking process, Procrustes analysis is usually adopted to transform the 3D data of the patient's pelvis collected by CT and the 2D data collected by X-ray. The rigid transformation matches the 3D points of interest of the patient and calculates the pose deviation between the two to complete the registration.

[0058] Existing shape alignment methods, such as Procrustes analysis, have certain effects in dealing with the rigid alignment of 3D point sets, but there are still the following deficiencies:

[0059] (1) Prone to local optimum

[0060] Procrustes analysis relies on least squares optimization. When the point correspondence is not perfectly matched or there is noise, it may converge to a local optimum solution, resulting in a large deviation in the final pose estimation.

[0061] (2) Poor robustness to mismatched points

[0062] In practical applications, 3D point sets often contain noise points, occluded points, or imperfectly matched points. The Procrustes method lacks a global optimization mechanism and is easily affected by mismatched points, resulting in a decrease in registration accuracy.

[0063] (3) Lack of global search ability

[0064] The traditional Procrustes method is based on analytical solutions or iterative optimization, and its search space is relatively limited. Especially in the case of scale changes or non-ideal initial alignment, it may be difficult to find the true global optimum solution.

[0065] Therefore, this application provides a method for aligning the pose of a 3D model with the pose of the human pelvis, aiming to solve the problems of local optimum, sensitivity to mismatches, and lack of global search ability existing in the traditional Procrustes analysis method.

[0066] The preferred embodiments of the present application will be described below in conjunction with the accompanying drawings of the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. And without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0067] Figure 2 It is a schematic flowchart of a method for aligning the pose of a three-dimensional model with the pose of a human pelvis provided by an embodiment of the present application. As Figure 2 shown, the method may include:

[0068] Step S210, obtaining the pelvic 2D data of the patient in minimally invasive pelvic surgery and the model pose of the three-dimensional model of the patient pre-constructed.

[0069] Among them, the pelvic 2D data may be image data collected from multiple perspectives of the patient's current pelvis by X-ray.

[0070] The three-dimensional model may be constructed from the pelvic 3D data collected by CT for the patient before the surgery.

[0071] It should be noted that the pelvic 2D data is image data collected from at least two perspectives.

[0072] Step S220, determining the initial positions of multiple interest points in the pelvic 2D data based on the pelvic 2D data, the pre-configured pelvic 3D data, and the interest point positions of multiple interest points in the pelvic 3D data.

[0073] Among them, the initial positions represent the two-dimensional positions of the respective interest points in the pelvic 2D data.

[0074] Specifically, before performing step S220, the method may include: the training process of the interest point tracking network, which may specifically include:

[0075] Marking multiple 3D interest points in the pelvic 3D data.

[0076] Constructing training data: The pelvic 3D data collected by CT is projected from different angles to generate multiple DRR images, and the 3D interest points generate 2D interest points according to the projection angle same as the angle of generating the DRR.

[0077] These 2D interest points and the DRR images are corresponding (for example, there are 3 interest points on a pelvic 3D data, and the generated 2D interest points are also three, corresponding to a DRR at a certain angle).

[0078] Model Training: The purpose of the model is to be able to input an X-Ray image (pelvic 2D data) and track the corresponding interest point positions. However, due to the small number of X-Ray images and the time-consuming nature of manually annotating a large number of interest point coordinates for training, DRR is used to simulate X-Ray for training. Therefore, the training samples are: DRR images, and the training labels are: DRR images with 2D interest points;

[0079] Using the training samples and training labels, the pre-trained interest point tracking network is trained to obtain a trained interest point tracking network.

[0080] In some embodiments, the Binary Cross Entropy Loss (BCELoss) is used as the loss function during the training of the interest point tracking network.

[0081] Typically in a clinical environment, the number of available multi-view X-ray images is limited, and the cost of manually annotating interest point labels is high, resulting in difficulty for the network to converge when directly using X-ray data for training and affecting the registration effect. To solve this problem, a style transfer module is introduced to use DRR images for network training and reduce the style difference between them and real X-rays. The DRR images are generated by projecting the pelvic 3D data collected by CT at different angles.

[0082] Specifically, this application uses DRR image pairs generated from different perspectives to train the interest point tracking network to make up for the training difficulties caused by insufficient clinical X-ray data. However, since there are significant style differences between the X-ray images obtained during surgery and the DRR images, the network trained directly with DRR is difficult to generalize to real X-ray images. Therefore, the style transfer module is used to align the styles of X-rays and DRRs, making the style of X-rays closer to that of DRRs, thereby enhancing the adaptability of the network to real X-ray images. The addition of style transfer effectively enhances the performance of the interest point tracking network and provides a feasible solution for 2D / 3D registration with limited data.

[0083] That is to say, since the pelvic 2D data are X-ray images, these images are two-dimensional and provide basic information about the bone structure. Combining Figure 3 As shown, to reduce the style difference between X-rays and DRR images, image-to-image conversion processing can be performed through Generative Adversarial Networks (GANs) to convert X-rays into a style closer to DRRs. By training a generator network to learn the mapping relationship from X-rays to DRRs and training a discriminator network to evaluate the authenticity of the converted images, the style transfer effect can be optimized; or, through Convolutional Neural Networks (CNNs), the relationship between the input image (X-ray image) and the target domain (such as DRR images) can be learned and mapped to achieve style unification.

[0084] After that, the trained interest point tracking network is used to process the pelvic 2D data to obtain the initial positions of multiple interest points.

[0085] Combined Figure 4 As shown, the interest point tracking network includes: an input layer, an encoder, a decoder, and an output layer connected in sequence;

[0086] Among them, A. The input layer is used to receive the pelvic 2D data.

[0087] B. The encoder is used to perform encoding processing on the pelvic 2D data;

[0088] The encoder includes: 5 convolutional blocks;

[0089] Each convolutional block includes a first convolutional layer, a first batch normalization layer, a first activation function, a second convolutional layer, a second batch normalization layer, a second activation function, an attention mechanism unit, and a third activation function connected in sequence. Among them, both the first convolutional layer and the second convolutional layer are 3×3 convolutional layers; the first activation function, the second activation function, and the third activation function are all ReLU activation functions.

[0090] Furthermore, the attention mechanism unit includes a channel attention module (Channel Attention) and a spatial attention module (Spatial Attention) connected in sequence.

[0091] Among them, the channel attention module includes a parallel global average pooling sub-module (GAP) and a global maximum pooling sub-module (GMP), a first convolutional sub-module and a second convolutional sub-module connected to the global average pooling sub-module in sequence, a third convolutional sub-module and a fourth convolutional sub-module connected to the global maximum pooling sub-module in sequence, and a fourth activation function connected to the second convolutional sub-module and the fourth convolutional sub-module;

[0092] Specifically, the global average pooling sub-module compresses the feature map into a 1x1xC vector; the global maximum pooling sub-module compresses the feature map into a 1x1xC vector. The first convolutional sub-module and the second convolutional sub-module, and the third convolutional sub-module and the fourth convolutional sub-module respectively perform convolutional operations on the results of GAP and GMP. The fourth activation function processes the convolutional results to generate channel weights and applies the channel weights to the original feature map (the feature map processed by the second activation function).

[0093] The spatial attention module includes a channel pooling sub-module, a fifth convolutional sub-module, and a fifth activation function connected in sequence.

[0094] Specifically, the channel pooling sub-module compresses the feature map into a 2xHxW vector, and the fifth convolution sub-module performs a convolution operation on the result of the channel pooling to obtain a convolution result. Subsequently, the convolution result is processed by the fifth activation function to generate a spatial weight, and the spatial weight is applied to the original feature map. Among them, the fifth convolution sub-module is a 7×7 convolutional layer.

[0095] C. The decoder includes multiple deconvolution layers.

[0096] The multiple deconvolution layers are used to gradually restore the resolution of the image. A ReLU activation function is also connected behind each deconvolution layer.

[0097] D. The output layer is used to output the image processed by the decoder. This image is in the form of a heat map, identifying the probability distribution of each point, so as to determine the exact position of the interest point.

[0098] Since the pelvic minimally invasive surgery navigation system has very high requirements for the real-time performance and accuracy of registration, the tracking network structure of the existing technology is complex, the training time is long, and the robustness is low. In order to improve the accuracy of interest point tracking and the robustness of the network, this application uses U-Net as the basic architecture and adds a Convolutional Block Attention Module (CBAM) to design an interest point tracking network. In order to guide the neural network to focus on the key areas of the image, a multi-scale Convolutional Block Attention Module is added to each layer of the network. The Convolutional Block Attention Module weights the input data in different dimensions through the channel attention mechanism and the spatial attention mechanism respectively, so as to enhance the network's ability to capture important information and reduce the interference of irrelevant information, thereby improving the performance of the model. Finally, it directly outputs the heat map form of the interest point tracking, and then obtains the position coordinates of the interest point, effectively extracts and utilizes the multi-scale features in the image, so as to improve the performance of the image processing task.

[0099] The network architecture is simple and clear, and can track interest points quickly and accurately, providing an efficient and lightweight solution for the actual application scenario.

[0100] After processing the pelvic 2D data (DRR image) with the trained interest point tracking network, combined with Figure 4 As shown, since the pelvic 2D data is a two-dimensional image collected at multiple angles, the initial positions of multiple interest points in the two-dimensional images (View1 X-ray and View2 X-ray) at each angle are obtained in step S220.

[0101] Step S230: Based on the triangulation method, process the initial positions of the interest points to obtain the target positions of the interest points.

[0102] Among them, the target position represents the three-dimensional position of each interest point in the patient's body;

[0103] Specifically, since each point of interest is collected from different perspectives, based on the positions of two observation points (the positions for obtaining 2D pelvic data), the distance between the two observation points is determined, and this distance is regarded as the baseline length. Using trigonometric functions and the measured angle data (the angles when obtaining 2D pelvic data), combined with the baseline length and the initial positions of each point of interest, the distance and direction of any point of interest relative to each observation point are calculated, and thus the exact target position of this point of interest can be determined.

[0104] Step S240: Process the point-of-interest positions of multiple points of interest and the target positions of each point of interest to obtain a target pose offset matrix.

[0105] Specifically, step S240-1: Based on the point-of-interest positions of multiple points of interest and the target positions of each point of interest, construct an alignment error function;

[0106] Define the set of point-of-interest positions of multiple points of interest in the 3D pelvic data collected by CT as:

[0107]

[0108] Define the set of point-of-interest positions of the three-dimensional positions of each point of interest in the patient's body as:

[0109]

[0110] Since the goal of shape alignment is to make X 3 and CT aligned through a rigid transformation matrix T (the target pose offset matrix, which includes a rotation matrix R ∈ SO(3) and a translation matrix t ∈ R ), thereby minimizing the alignment error and making their poses basically consistent in the same coordinate system;

[0111] This alignment error function can be expressed as:

[0112]

[0113] It can also be expressed as:

[0114]

[0115] It can be understood that based on the point-of-interest positions of multiple points of interest and the target positions of each point of interest, multiple pose offset matrices can be determined, thereby constructing a pose offset matrix queue.

[0116] Step S240-2: Divide the pose offset matrix queue to obtain multiple initial spaces.

[0117] Specifically, a number of initial spaces are generally evenly divided according to the projection angle of the DRR image; for example, R ∈ [-8, 8] is evenly divided into 8 initial spaces.

[0118] Step S240-3: Recursively search in multiple initial spaces based on the configured initial transformation search range to obtain the initial space with the minimum alignment error.

[0119] The initial transformation search range is: and t ∈ [-15, 15] mm.

[0120] After that, if the difference between the upper error bound and the lower error bound of the initial space is less than the configured error threshold, the initial pose offset matrix corresponding to the initial space with the minimum alignment error is determined as the target pose offset matrix. Otherwise, execute step S240-4.

[0121] Step S240-4: Divide the initial space with the minimum alignment error to obtain multiple sub-spaces.

[0122] Specifically, according to the number of the above initial spaces, the initial space with the minimum alignment error is divided to obtain the same number of sub-spaces.

[0123] Step S240-5: Determine the sub-space with the minimum alignment error among multiple sub-spaces.

[0124] After that, if the difference between the upper error bound and the lower error bound of the sub-space with the minimum alignment error is less than the configured error threshold, the initial pose offset matrix corresponding to the sub-space with the minimum alignment error is determined as the target pose offset matrix.

[0125] This process is a recursive search. Each time, the sub-space with the minimum alignment error is taken out from the queue. If the difference between the upper error bound and the lower error bound is less than the error threshold, the search stops.

[0126] If the difference between the upper error bound and the lower error bound of the sub-space with the minimum alignment error is not less than the configured error threshold, and the lower error bound is not greater than the minimum alignment error, the sub-space with the minimum alignment error is divided to obtain multiple new sub-spaces, and the multiple new sub-spaces are used as the current sub-spaces, and return to execute the step: Determine the sub-space with the minimum alignment error among multiple sub-spaces.

[0127] If the difference between the upper error bound and the lower error bound of the sub-space with the minimum alignment error is not less than the configured error threshold, and the lower error bound is greater than the minimum alignment error, the sub-space with the minimum alignment error is deleted among multiple sub-spaces to obtain a new multiple sub-spaces, and return to execute the step: Determine the sub-space with the minimum error among multiple sub-spaces.

[0128] This process is pruning optimization, that is, if the lower bound of the error of a certain subspace is greater than the current optimal solution (the minimum alignment error), then prune it to avoid unnecessary calculations.

[0129] Furthermore, the upper bound of the error is determined as follows:

[0130] A. Determine the first distance between the first centroid and the bounding box of multiple points of interest in the pelvic 3D data based on the first centroid of multiple points of interest in the pelvic 3D data;

[0131] Determine the second distance between the second centroid and the bounding box of each point of interest in the patient's body based on the second centroid of each point of interest in the patient's body;

[0132] Determine the average value of the first distance and the second distance as the upper bound of the error.

[0133] B. The lower bound of the error is equal to the minimum alignment error, and its calculation method is as follows:

[0134]

[0135] In one example: Initialize the optimization queue q ← 0, the transformation matrix search space T is defined by and t ∈ [-15, 15] mm, R * ← I, t * ← 0, E * ← +∞, put T into q;

[0136] When the queue q is not empty: Read out the initial space sub_T with the minimum error from the queue, and the corresponding lower bound of the error is E T ;

[0137] If the difference between the upper bound of the error and the lower bound of the error is less than the threshold E * - E T <δ, then exit the algorithm;

[0138] Otherwise, divide the initial space sub_T into 8 subspaces, and for each subspace, execute:

[0139] Calculate

[0140]

[0141] Compute E T

[0142] If E T > E * then, discard this subspace and repeat the above steps.

[0143] Put the current T * into the queue and return T * , E * .

[0144] This method overcomes the problems of local optimization methods (such as Procrustes) being prone to local optima and being sensitive to mismatched points through a global search strategy, ensuring that the optimal rigid transformation matrix is found globally, thereby achieving high-precision 3D point set registration. That is, by constructing a search tree, bounds estimation, and pruning strategies, it is guaranteed that the final solution is the global optimal solution, thus avoiding local optimal problems and improving the alignment accuracy. This application can still correctly estimate the optimal pose even when the initial match is not ideal, and can still accurately estimate the shape alignment even when the input point set contains noise or partial occlusion. In addition, an efficient pruning strategy is designed to dynamically set bounds during the search process, eliminating impossible solutions in advance, thereby reducing the computational overhead. Improve the computational efficiency and efficiently complete the shape alignment task.

[0145] Step S250: Adjust the model pose according to the target pose offset matrix to obtain the target pose.

[0146] Among them, the target pose represents the current pelvic pose of the patient, so that the 3D model pose is aligned with the patient's pelvic pose.

[0147] Specifically, the target pose offset matrix shows the spatial difference between the current pelvic position of the patient and the 3D model. According to the target pose offset matrix, the position and orientation of the 3D model can be adjusted to match the actual situation of the patient. Specifically, it is to transform the 3D model by applying the target pose offset matrix so that its final placement position is exactly the same as the actual position and orientation of the patient's pelvis. The purpose of doing this is to ensure that the 3D model can accurately reflect the patient's current anatomical structure and provide precise guidance for the surgery.

[0148] The present application provides a method for aligning the pose of a 3D model with the pose of a human pelvis. The method includes: obtaining the 2D data of the pelvis of a patient in a minimally invasive pelvic surgery and the model pose of the 3D model of the patient pre-constructed; determining a plurality of initial pose offset matrices through the interest point positions of a plurality of interest points in the pre-configured 3D data of the pelvis and the target positions of each interest point in the patient's body; dividing the pose offset matrix queue composed of a plurality of initial pose offset matrices to obtain a plurality of initial spaces; recursively searching in the plurality of initial spaces based on the configured initial transformation search range to obtain the initial space with the minimum alignment error; the minimum alignment error is calculated through a constructed alignment error function; if the difference between the upper error bound and the lower error bound of the initial space with the minimum alignment error is less than the configured error threshold, determining the initial pose offset matrix corresponding to the initial space with the minimum alignment error as the target pose offset matrix; adjusting the model pose according to the target pose offset matrix to obtain the target pose, so as to align the model pose of the 3D model with the pose of the pelvis of the patient. The present application adopts a step-by-step subdivision search strategy to find the global optimal solution, realizes the branch and pruning steps, divides any subset into smaller subsets, and calculates the upper and lower bounds of the alignment error used to prune each subset. Then, by excluding the subspaces that may not contain the optimal solution in the search process, the scale of the search space is reduced, the algorithm efficiency is improved, and it has higher adaptability and generality for the shape alignment problem.

[0149] Corresponding to the above method, an embodiment of the present application further provides an apparatus for aligning the pose of a 3D model with the pose of a human pelvis, as Figure 5 shown. The apparatus includes:

[0150] An obtaining unit 510, configured to obtain the 2D data of the pelvis of a patient in a minimally invasive pelvic surgery and the model pose of the 3D model of the patient pre-constructed;

[0151] A determining unit 520, configured to determine a plurality of initial pose offset matrices through the interest point positions of a plurality of interest points in the pre-configured 3D data of the pelvis and the target positions of each interest point in the patient's body;

[0152] A dividing unit 530, configured to divide the pose offset matrix queue composed of a plurality of initial pose offset matrices to obtain a plurality of initial spaces;

[0153] A searching unit 540, configured to recursively search in the plurality of initial spaces based on the configured initial transformation search range to obtain the initial space with the minimum alignment error; the minimum alignment error is calculated through a constructed alignment error function;

[0154] The determining unit 520 is further configured to determine the initial pose offset matrix corresponding to the initial space with the minimum alignment error as the target pose offset matrix if the difference between the upper error bound and the lower error bound of the initial space with the minimum alignment error is less than the configured error threshold;

[0155] An adjusting unit 550 is configured to adjust the model pose according to the target pose offset matrix to obtain a target pose, where the target pose represents the current pelvic pose of the patient, so as to align the model pose of the three-dimensional model with the pelvic pose of the patient.

[0156] The functions of the functional units of the alignment device for the three-dimensional model pose and the human pelvic pose provided in the above embodiments of the present application can be implemented by the above method steps. Therefore, the specific working processes and beneficial effects of each unit in the alignment device for the three-dimensional model pose and the human pelvic pose provided in the embodiments of the present application will not be elaborated here.

[0157] The embodiments of the present application further provide an electronic device, as Figure 6 shown, including a processor 610, a communication interface 620, a memory 630, and a communication bus 640. Among them, the processor 610, the communication interface 620, and the memory 630 complete communication with each other through the communication bus 640.

[0158] The memory 630 is used to store a computer program;

[0159] When the processor 610 is configured to execute the program stored in the memory 630, the following steps are implemented:

[0160] Obtain the pelvic 2D data of the patient during minimally invasive pelvic surgery and the model pose of the three-dimensional model of the patient pre-constructed;

[0161] Determine a plurality of initial pose offset matrices through the interest point positions of a plurality of interest points in the pre-configured pelvic 3D data and the target positions of each interest point in the patient's body;

[0162] Divide the pose offset matrix queue composed of a plurality of initial pose offset matrices to obtain a plurality of initial spaces;

[0163] Recursively search in a plurality of initial spaces based on the configured initial transformation search range to obtain the initial space with the minimum alignment error; the minimum alignment error is calculated through the constructed alignment error function;

[0164] If the difference between the upper error bound and the lower error bound of the initial space with the minimum alignment error is less than the configured error threshold, determine the initial pose offset matrix corresponding to the initial space with the minimum alignment error as the target pose offset matrix;

[0165] Adjust the model pose according to the target pose offset matrix to obtain a target pose, which represents the current pelvic pose of the patient, so as to align the model pose of the three-dimensional model with the pelvic pose of the patient.

[0166] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0167] The communication interface is used for communication between the above-mentioned electronic device and other devices.

[0168] The memory may include a Random Access Memory (RAM), and may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0169] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0170] Since the implementation manners and beneficial effects of the various devices of the electronic device in the above embodiments can be realized by referring to the steps in the embodiments shown in Figure 2 Therefore, the specific working process and beneficial effects of the electronic device provided by the embodiments of the present application will not be repeated here.

[0171] In another embodiment provided by the present application, a computer-readable storage medium is further provided. Instructions are stored in the computer-readable storage medium. When it runs on a computer, the computer is enabled to execute a method for aligning the pose of a three-dimensional model with the pose of a human pelvis described in any one of the above embodiments.

[0172] In another embodiment provided by the present application, a computer program product containing instructions is further provided. When it runs on a computer, the computer is enabled to execute a method for aligning the pose of a three-dimensional model with the pose of a human pelvis described in any one of the above embodiments.

[0173] Those skilled in the art should understand that the embodiments in the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the embodiments in the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments in the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] The embodiments in the present application are described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments in the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0175] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the functions in the process Figure 1steps of one or more processes and / or blocks Figure 1 steps of the functions specified in one or more blocks

[0177] Unless otherwise defined, technical terms or scientific terms used in this application shall have the ordinary meanings understood by those of ordinary skill in the art to which this invention pertains. The terms "first", "second", and similar terms used in this application do not denote any order, quantity, or importance, but are only used to distinguish different components. Words such as "comprising" or "including" mean that the elements or items appearing before the word cover the elements or items listed after the word and their equivalents, without excluding other elements or items. Words such as "connected", "coupled", or "linked" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right", etc. are only used to indicate relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0178] Although the preferred embodiments in the embodiments of this application have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concept. Therefore, the embodiments of this application are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of this application.

[0179] Obviously, those skilled in the art can make various changes and variations to the embodiments in the embodiments of this application without departing from the spirit and scope of the embodiments in the embodiments of this application. Thus, if these modifications and variations of the embodiments in the embodiments of this application fall within the scope of the embodiments of this application and their equivalent technologies, the embodiments of this application also intend to include these changes and variations.

Claims

1. A method for aligning the pose of a three-dimensional model with the pose of a human pelvis, characterized in that The method includes: Obtaining the pelvic 2D data of the patient during minimally invasive pelvic surgery and the model pose of the pre-constructed three-dimensional model of the patient; Determining a plurality of initial pose offset matrices based on the interest point positions of a plurality of interest points in the pre-configured pelvic 3D data and the target positions of each interest point in the patient's body; Dividing the pose offset matrix queue composed of a plurality of initial pose offset matrices to obtain a plurality of initial spaces; Recursively searching in a plurality of initial spaces based on the configured initial transformation search range to obtain the initial space with the minimum alignment error; the minimum alignment error is calculated by the constructed alignment error function; If the difference between the error upper bound and the error lower bound of the initial space with the minimum alignment error is less than the configured error threshold, determining the initial pose offset matrix corresponding to the initial space with the minimum alignment error as the target pose offset matrix; Adjusting the model pose according to the target pose offset matrix to obtain the target pose, where the target pose represents the current pelvic pose of the patient, so as to align the model pose of the three-dimensional model with the pelvic pose of the patient.

2. The method according to claim 1, wherein The construction of the alignment error function includes: Determining the initial positions of a plurality of interest points in the pelvic 2D data based on the pelvic 2D data, the pelvic 3D data, and the interest point positions of a plurality of interest points in the pelvic 3D data; Processing the initial positions of each interest point based on the triangulation method to obtain the target positions of each interest point; Constructing an alignment error function based on the interest point positions of a plurality of interest points and the target positions of each interest point.

3. The method according to claim 2, wherein The alignment error function is: Among them, T* is the target pose offset matrix, T is the position and pose offset matrix, and X CT is the set of the interest point positions of multiple interest points, and is the set of the target positions of each interest point in the patient's body.

4. The method according to claim 1, characterized in that The method further includes: If the difference between the error upper bound and the error lower bound of the initial space with the minimum alignment error is not less than the configured error threshold, and the error lower bound is not greater than the minimum alignment error, dividing the initial space with the minimum error to obtain a plurality of sub-spaces; Determining the sub-space with the minimum alignment error among the plurality of sub-spaces; If the difference between the error upper bound and the error lower bound of the sub-space with the minimum alignment error is less than the configured error threshold, determining the initial pose offset matrix corresponding to the sub-space with the minimum alignment error as the target pose offset matrix; If the difference between the error upper bound and the error lower bound of the sub-space with the minimum alignment error is not less than the configured error threshold, and the error lower bound is not greater than the minimum alignment error, dividing the sub-space with the minimum alignment error to obtain a plurality of new sub-spaces, and taking the plurality of new sub-spaces as the current sub-spaces, and returning to execute the step: determining the sub-space with the minimum alignment error among the plurality of sub-spaces.

5. The method according to claim 4, wherein The method further includes: If the difference between the error upper bound and the error lower bound of the sub-space with the minimum alignment error is not less than the configured error threshold, and the error lower bound is greater than the minimum alignment error, deleting the sub-space corresponding to the minimum alignment error among the plurality of sub-spaces to obtain a new plurality of sub-spaces, and returning to execute the step: determining the sub-space with the minimum error among the plurality of sub-spaces.

6. The method according to claim 4, wherein The determination method of the error upper bound is: Based on the first centroid of a plurality of interest points in the pelvic 3D data, determining the first distance between the first centroid and the bounding box of the plurality of interest points in the pelvic 3D data; Determine a second distance between the second centroid and a bounding box of each point of interest in a patient's body based on the second centroid of each point of interest in the patient's body; Determine the average value of the first distance and the second distance as the upper bound of the error.

7. The method according to claim 4, wherein The lower bound of the error is equal to the minimum alignment error.

8. An alignment device for the pose of a three-dimensional model and the pose of a human pelvis, characterized in that, The apparatus includes: An acquisition unit, configured to acquire 2D data of a patient's pelvis and a model pose of a pre-constructed three-dimensional model of the patient during minimally invasive pelvic surgery; A determination unit, configured to determine a plurality of initial pose offset matrices based on the positions of a plurality of points of interest in pre-configured 3D pelvic data and the target positions of each point of interest in the patient's body; A segmentation unit, configured to segment a pose offset matrix queue composed of a plurality of initial pose offset matrices to obtain a plurality of initial spaces; A search unit, configured to recursively search in a plurality of initial spaces based on a configured initial transformation search range to obtain an initial space with a minimum alignment error; the minimum alignment error is calculated by a constructed alignment error function; The determination unit is further configured to, if the difference between the upper bound of the error and the lower bound of the error of the initial space with the minimum alignment error is less than a configured error threshold, determine the initial pose offset matrix corresponding to the initial space with the minimum alignment error as the target pose offset matrix; An adjustment unit, configured to adjust the model pose according to the target pose offset matrix to obtain a target pose, where the target pose represents the current pelvic pose of the patient, so as to align the model pose of the three-dimensional model with the pelvic pose of the patient.

9. An electronic device, characterized in that, The electronic device includes a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory is used to store a computer program; The processor is configured to implement the method steps according to any one of claims 1-7 when executing the program stored on the memory.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps according to any one of claims 1-7 are implemented.

Images