Human body pelvis posture recognition method and device, electronic equipment and medium

Abstract

The application provides a human pelvis pose recognition method and device, electronic equipment and medium, and relates to the field of medical image registration. The method comprises the following steps: acquiring the pelvis 2D data and model pose of a patient in a minimally invasive pelvis surgery; inputting the pelvis 2D data into a trained interest point tracking network for processing to obtain the first positions of multiple interest points; processing the first positions of the interest points based on the triangulation method to obtain the second positions of the interest points, and then determining a pose offset matrix; and adjusting the model pose based on the pose offset matrix. The method proposed in the application can be understood as a multi-view 2D / 3D image registration method in pelvis minimally invasive surgery navigation based on transfer learning. A lightweight interest point tracking network combining U-Net and an attention mechanism is designed, and a style transfer module is introduced to effectively solve the problem of insufficient data and improve the performance of the network on real X-rays.

Description

Technical Field

[0001] This application relates to the field of medical image registration, and more specifically, to a method, device, electronic device, and medium for recognizing the posture of the human pelvis. Background Technology

[0002] Pelvic fractures are a serious trauma and have always been a significant challenge for orthopedic surgeons. While traditional open surgery is effective in treating fractures, it involves significant intraoperative trauma and slow recovery. With advancements in medical technology, minimally invasive pelvic surgery has gradually become an emerging method for treating pelvic fractures due to its advantages such as less trauma and faster recovery. Currently, the successful implementation of minimally invasive surgery often relies on surgical navigation systems, making accurate positioning and navigation crucial. One of the core technologies in minimally invasive pelvic surgical navigation systems is 2D / 3D registration technology, which provides real-time and accurate three-dimensional positioning for minimally invasive pelvic surgery.

[0003] The main task of 2D / 3D registration is to find the optimal 3D transformation to align and register the 3D data acquired by preoperative CT with the intraoperative 2D X-ray images, establishing a correspondence between the preoperative 3D data and the intraoperative 2D X-ray. This allows both to represent the position and shape of the object in the same spatial coordinate system, indirectly providing doctors with real-time 3D pose information and guidance for the patient during surgery, thereby improving the safety and effectiveness of the operation. Traditional methods are usually based on optimization and iteration, which are computationally complex, time-consuming, and prone to getting trapped in local optima, making it difficult to meet the requirements of clinical accuracy. Currently, deep learning-based methods mostly use single-view registration. However, single-view registration itself suffers from ill-posedness, leading to a high registration failure rate and affecting accuracy and robustness. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, electronic device and medium for recognizing human pelvic posture, so as to solve the above-mentioned problems existing in the prior art and improve the registration efficiency and registration accuracy.

[0005] Firstly, a method for recognizing the human pelvic posture is provided, which is applied to the tracking and positioning module of a minimally invasive pelvic surgery navigation system. This method may include:

[0006] Acquire 2D data of the pelvis of a patient during minimally invasive pelvic surgery and the model pose of a pre-constructed 3D model of the patient; wherein, the 2D data of the pelvis is data acquired from multiple perspectives of the patient's current pelvis via X-ray.

[0007] The 2D data of the pelvis is input into a trained interest point tracking network for processing to obtain the first position of multiple interest points; the first position represents the two-dimensional position of each interest point in the 2D data of the pelvis.

[0008] Based on triangulation, the first position of each point of interest is processed to obtain the second position of each point of interest; the second position represents the three-dimensional position of each point of interest in the patient's body.

[0009] Based on the second position of each point of interest, determine the pose offset matrix;

[0010] Based on the pose offset matrix, the model pose is adjusted to obtain the target pose, which represents the patient's current pelvic pose.

[0011] In one possible implementation, the interest point tracking network includes: an input layer, an encoder, a decoder, and an output layer connected in sequence;

[0012] The encoder includes 5 convolutional blocks;

[0013] Each convolutional block includes a first convolutional layer, a first batch normalization layer, a first activation function, a second convolutional layer, a second batch normalization layer, a second activation function, an attention mechanism unit, and a third activation function, all connected in sequence.

[0014] In one possible implementation, the attention mechanism unit includes a channel attention module and a spatial attention module connected in sequence.

[0015] In one possible implementation, the channel attention module includes a parallel global average pooling submodule and a global max pooling submodule, a first convolutional submodule and a second convolutional submodule connected sequentially to the global average pooling submodule, a third convolutional submodule and a fourth convolutional submodule connected sequentially to the global max pooling submodule, and a fourth activation function connected to the second convolutional submodule and the fourth convolutional submodule.

[0016] The spatial attention module includes a channel pooling submodule, a fifth convolution submodule, and a fifth activation function connected in sequence.

[0017] In one possible implementation, the training process of the interest point tracking network includes:

[0018] The configured 3D data of the patient's pelvis is processed to obtain 2D data of the target pelvis from different angles; multiple points of interest are marked in the 3D data of the pelvis.

[0019] Multiple target pelvic 2D data were used as training samples, and target pelvic 2D data carrying points of interest were used as training labels.

[0020] The interest point tracking network is trained using the training samples and the training labels to obtain a trained interest point tracking network.

[0021] In one possible implementation, the configured 3D data of the patient's pelvis is processed to obtain 2D data of the target pelvis from different angles, including:

[0022] The 3D data of the pelvis is processed to obtain initial 2D data of the pelvis from different angles;

[0023] For any initial pelvic 2D data from any angle, style transfer techniques are used to convert X-ray images into target pelvic 2D data with a consistent style, thereby reducing the style difference between X-ray and initial pelvic 2D data.

[0024] Secondly, a device for recognizing the position of the human pelvis is provided, which is applied to the tracking and positioning module of a minimally invasive pelvic surgery navigation system. This device may include:

[0025] The acquisition unit is used to acquire 2D data of the patient's pelvis and the model pose of a pre-constructed 3D model of the patient during minimally invasive pelvic surgery; wherein, the 2D data of the pelvis is data acquired from multiple perspectives of the patient's current pelvis via X-ray.

[0026] The processing unit is used to input the pelvic 2D data into a trained interest point tracking network for processing to obtain the first position of multiple interest points; the first position represents the two-dimensional position of each interest point in the pelvic 2D data.

[0027] Furthermore, based on triangulation, the first position of each point of interest is processed to obtain the second position of each point of interest; the second position represents the three-dimensional position of each point of interest within the patient's body.

[0028] The determining unit is used to determine the pose offset matrix based on the second position of each interest point;

[0029] An adjustment unit is used to adjust the model pose based on the pose offset matrix to obtain a target pose, which represents the patient's current pelvic pose.

[0030] Thirdly, an electronic device is provided, which includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0031] Memory, used to store computer programs;

[0032] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect above.

[0033] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the steps of any of the methods described in the first aspect above.

[0034] This application provides a method for recognizing the pose of the human pelvis. The method includes: acquiring 2D data of the pelvis of a patient undergoing minimally invasive pelvic surgery and the pose of a pre-constructed 3D model of the patient; inputting the 2D pelvic data into a trained interest point tracking network for processing to obtain the first positions of multiple interest points; processing the first positions of each interest point based on triangulation to obtain the second positions of each interest point; determining a pose offset matrix based on the second positions of each interest point; and adjusting the model pose based on the pose offset matrix to obtain the target pose. The method proposed in this application can be understood as a multi-view approach in style transfer-based navigation for minimally invasive pelvic surgery. Figure 2 A D / 3D image registration method was developed, which designed a lightweight interest point tracking network combining U-Net with an attention mechanism. The style transfer module introduced can effectively solve the problem of insufficient data and improve the network's performance on real X-rays. Attached Figure Description

[0035] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 A system architecture diagram for a method for recognizing human pelvic pose provided in an embodiment of this application;

[0037] Figure 2 A flowchart illustrating a method for recognizing the human pelvic posture provided in an embodiment of this application;

[0038] Figure 3 A schematic diagram of the transfer learning process provided in an embodiment of this application;

[0039] Figure 4 This is a schematic diagram of the structure of the point of interest tracking network provided in the embodiments of this application;

[0040] Figure 5 A schematic diagram of the structure of a human pelvic posture recognition device provided in an embodiment of this application;

[0041] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0043] The method for recognizing human pelvic posture provided in this application embodiment can be applied to... Figure 1 In the system architecture of the pelvic minimally invasive surgery navigation system shown, such as Figure 1 As shown, the system may include:

[0044] Image Acquisition and Processing Module: This module is responsible for acquiring preoperative and intraoperative medical imaging data of the patient, such as preoperative CT (computed tomography), MRI (magnetic resonance imaging), and intraoperative X-rays. It is also responsible for generating DRR images from the 3D data acquired by CT for subsequent use. This image data is used to construct a 3D model of the patient and perform registration operations, providing detailed anatomical information for the surgery.

[0045] Surgical planning module: Based on the acquired imaging data, surgeons can use surgical planning software to create virtual surgical plans, including determining the optimal surgical path, simulating the location and size of implants, etc. This helps to develop the optimal surgical plan before the actual operation.

[0046] Tracking and Positioning Module: This system uses optical or electromagnetic trackers to monitor the position of surgical instruments relative to the patient's body in real time. By matching the positional information in physical space with a pre-built 3D model, it can provide surgeons with precise operational guidance. This application is used in this module to acquire the model pose of a pre-built 3D model of the patient and 2D data of the patient's pelvis during minimally invasive pelvic surgery, in order to execute the human pelvic pose recognition method provided in this application.

[0047] Display and User Interface: An intuitive and easy-to-use user interface is essential for operating such systems. It typically includes touchscreen displays, head-mounted displays, or other forms of visualization tools to allow surgeons to view detailed images of the surgical area and information on surgical planning.

[0048] Assistive robots: may also include one or more assistive robotic arms that can perform precise actions under the control of a doctor, such as cutting, drilling, or placing implants.

[0049] Safety and Quality Control Module: To ensure the safety and effectiveness of the surgery, the system also includes a series of safety measures, such as error detection, boundary alarms, and real-time feedback mechanisms, to help doctors avoid potential operational errors.

[0050] Pelvic fractures are a serious trauma and have always been a significant challenge for orthopedic surgeons. While traditional open surgery is effective in treating fractures, it involves significant intraoperative trauma and slow recovery. With advancements in medical technology, minimally invasive pelvic surgery has gradually become an emerging method for treating pelvic fractures due to its advantages such as less trauma and faster recovery. Ensuring accurate positioning and navigation is crucial for the success of minimally invasive pelvic surgery and patient safety. Minimally invasive pelvic surgical navigation systems typically rely on 2D / 3D registration technology to provide real-time, accurate three-dimensional positioning for minimally invasive pelvic surgery.

[0051] The main task of 2D / 3D registration is to find the optimal 3D transformation to minimize the misalignment between the 2D image and the 3D volume. Traditional methods typically rely on optimization, which is both laborious and time-consuming, and often fails to achieve high registration accuracy. With the development of deep learning technology in image registration, some learning-based strategies have been applied to 2D / 3D registration. Initially, learning-based strategies focused on single-view registration, achieving high accuracy, but the results still need improvement in complex situations. Subsequently, multi-view registration was gradually introduced, but problems remain regarding data requirements, computational resources, and registration accuracy.

[0052] Furthermore, existing technologies suffer from complex network structures, poor real-time performance, and poor registration results when data is insufficient. The tracking network proposed by Haofu Liao et al. is built on the Siamese architecture, with each branch having a U-Net-like structure, and requires simultaneous training with both DRR and X-ray images as input. This results in a complex network structure, large amounts of input data, long training time, and poor real-time registration performance. Additionally, this method uses a large number of labeled X-ray images for training, while clinically available paired labeled X-ray images from different perspectives are very limited, failing to meet the network training requirements. Furthermore, manually labeling interest points is time-consuming and labor-intensive. This necessitates training with DRR and testing on X-ray data. However, the significant style differences between DRR and X-ray mean that networks trained with DRR cannot maintain training accuracy when applied to X-ray data. These issues significantly impact the accuracy of pelvic image registration, leading to insufficient registration precision and poor model generalization ability.

[0053] Therefore, this application provides a method for recognizing the human pelvic posture, which solves the above-mentioned problems existing in the prior art and can improve the registration efficiency and registration accuracy.

[0054] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0055] Figure 2 This is a flowchart illustrating a method for recognizing the human pelvic posture provided in an embodiment of this application. Figure 2 As shown, the method may include:

[0056] Step S210: Obtain the 2D data of the patient's pelvis and the model pose of the pre-constructed 3D model of the patient during minimally invasive pelvic surgery.

[0057] Among them, pelvic 2D data can be image data of the patient's current pelvis acquired from multiple perspectives using X-ray.

[0058] The 3D model can be constructed using the 3D data collected from the patient via CT scan before surgery.

[0059] It should be noted that the pelvic 2D data consists of image data acquired from at least two perspectives.

[0060] Step S220: Input the 2D data of the pelvis into the trained interest point tracking network for processing to obtain the first position of each interest point.

[0061] The first position represents the two-dimensional position of each point of interest in the pelvic 2D data.

[0062] Before performing step S220, the method may include: the training process of the interest point tracking network, specifically including:

[0063] Multiple 3D points of interest were labeled in the pelvic 3D data.

[0064] Construction of training data: 3D pelvic data acquired by CT is projected from different angles to generate multiple DRR images. 3D interest points are generated into 2D interest points according to the same projection angle as the angle from which the DRR is generated.

[0065] These 2D points of interest correspond to DRR images (for example, if there are 3 points of interest in a 3D pelvic image, there will also be 3 2D points of interest generated, corresponding to a DRR image at a certain angle).

[0066] Model Training: The goal of the model is to be able to track the location of the corresponding interest points from an input X-ray image. However, due to the limited number of X-ray images and the time-consuming nature of manually annotating a large number of interest point coordinates for training, DRR is used to simulate X-rays for training. Therefore, the training samples are: DRR images, and the training labels are: DRR images carrying 2D interest points.

[0067] In short: Process 3D pelvic data to obtain DRR images from different angles and DRR images with points of interest; for any DRR image and DRR image with points of interest, use the DRR image as a training sample and the DRR image with points of interest as a training label.

[0068] The pre-trained interest point tracking network is trained using training samples and training labels to obtain a well-trained interest point tracking network.

[0069] In some embodiments, the interest point tracking network uses BCELoss as the loss function during training.

[0070] In clinical settings, the number of available multi-view X-ray images is typically limited, and manually labeling points of interest is costly, making it difficult for networks to converge when X-ray data is directly used for training, thus affecting registration accuracy. To address this issue, a style transfer module is introduced, utilizing DRR images for network training and reducing style differences between them and real X-rays. The DRR images are generated from CT-acquired 3D pelvic data through projection at different angles.

[0071] Specifically, this application uses DRR image pairs generated from different viewpoints to train the interest point tracking network to compensate for the training difficulties caused by insufficient clinical X-ray data. However, since there are significant stylistic differences between X-ray images acquired during surgery and DRR images, networks trained directly using DRR images are difficult to generalize to real X-ray images. Therefore, a style transfer module is used to align the styles of X-ray and DRR images, making the style of X-ray images closer to that of DRR images, thereby improving the network's adaptability to real X-ray images. The addition of style transfer effectively enhances the performance of the interest point tracking network, providing a feasible solution for 2D / 3D registration with limited data.

[0072] In other words, since pelvic 2D data are X-ray images, these images are two-dimensional and provide basic information about the skeletal structure, combined with... Figure 3As shown, to reduce the style difference between X-ray and DRR images, generative adversarial networks (GANs) can be used to perform image-to-image transformation, converting X-ray images into a style closer to DRR. A generator network is trained to learn the mapping relationship from X-ray to DRR, and a discriminator network is trained to evaluate the realism of the transformed image, thereby optimizing the style transfer effect. Alternatively, convolutional neural networks (CNNs) can be used to learn and map the relationship between the input image (X-ray image) and the target domain (such as DRR image), thereby achieving style unification.

[0073] Then, the trained interest point tracking network was used to process the pelvic 2D X-Ray data to obtain the first positions of multiple interest points.

[0074] Combination Figure 4 As shown, the interest point tracking network includes an input layer, an encoder, a decoder, and an output layer connected in sequence.

[0075] Among them, A, the input layer, is used to receive 2D data of the pelvis.

[0076] B. Encoder, used to encode 2D data of the pelvis;

[0077] The encoder includes: 5 convolutional blocks;

[0078] Each convolutional block includes a first convolutional layer, a first batch normalization layer, a first activation function, a second convolutional layer, a second batch normalization layer, a second activation function, an attention mechanism unit, and a third activation function, all connected in sequence. The first and second convolutional layers are both 3×3 convolutional layers; the first, second, and third activation functions are all ReLU activation functions.

[0079] Furthermore, the attention mechanism unit includes a channel attention module and a spatial attention module connected in sequence.

[0080] The channel attention module includes parallel global average pooling (GAP) and global max pooling (GMP) submodules, a first convolutional submodule and a second convolutional submodule connected in sequence to the global average pooling submodule, a third convolutional submodule and a fourth convolutional submodule connected in sequence to the global max pooling submodule, and a fourth activation function connected to the second convolutional submodule and the fourth convolutional submodule.

[0081] Specifically, the global average pooling submodule compresses the feature map into a 1x1xC vector; the global max pooling submodule compresses the feature map into a 1x1xC vector. The first and second convolutional submodules, as well as the third and fourth convolutional submodules, perform convolution operations on the results of GAP and GMP, respectively. The fourth activation function processes the convolution results, generates channel weights, and applies these channel weights to the original feature map (the feature map processed by the second activation function).

[0082] The spatial attention module consists of a channel pooling submodule, a fifth convolution submodule, and a fifth activation function connected in sequence.

[0083] Specifically, the channel pooling submodule compresses the feature map into a 2xHxW vector, and then performs a convolution operation on the channel pooling result through the fifth convolution submodule to obtain the convolution result. Next, the convolution result is processed by the fifth activation function to generate spatial weights, which are then applied to the original feature map. The fifth convolution submodule is a 7×7 convolutional layer.

[0084] C. The decoder consists of multiple deconvolutional layers.

[0085] Multiple deconvolutional layers are used to progressively restore the image resolution. Each deconvolutional layer is also followed by a ReLU activation function.

[0086] D. Output layer: This layer outputs the image processed by the decoder. This image is in heatmap form, identifying the probability distribution of each point, thus determining the exact location of the point of interest.

[0087] Because pelvic minimally invasive surgery navigation systems require extremely high real-time performance and accuracy in registration, existing tracking networks are complex, time-consuming to train, and exhibit low robustness. To improve the accuracy and robustness of point of interest (POI) tracking, this application uses U-Net as the basic architecture and incorporates a hybrid attention mechanism (CBAM) to design an POI tracking network. To guide the neural network to focus on key regions of the image, a multi-scale hybrid attention mechanism is added to each layer. This mechanism weights the input data across different dimensions using channel and spatial attention mechanisms, enhancing the network's ability to capture important information and reducing interference from irrelevant information, thereby improving model performance. Finally, it directly outputs a heatmap of POI tracking, obtaining the location coordinates of the POIs and effectively extracting and utilizing multi-scale features in the image, thus improving the performance of image processing tasks.

[0088] The network architecture is simple and clear, and can quickly and accurately track points of interest, providing an efficient and lightweight solution for practical application scenarios.

[0089] After processing the 2D pelvic data using a pre-trained interest point tracking network, it is combined with... Figure 4 As shown, since the pelvic 2D data is a two-dimensional image acquired from multiple angles, step S220 obtains the first position of multiple points of interest in the two-dimensional images (View1 X-ray and View2 X-ray) from each angle.

[0090] Step S230: Based on the triangulation method, process the first position of each point of interest to obtain the second position of each point of interest.

[0091] The second position represents the three-dimensional position of each point of interest within the patient's body;

[0092] Specifically, since each point of interest (POI) is acquired from a different perspective, the distance between the two observation points (where the 2D pelvic data was obtained) is determined based on their positions and is considered the baseline length. Using trigonometric functions and measured angle data (the angles used when acquiring the 2D pelvic data), combined with the baseline length and the first position of each POI, the distance and direction of any POI relative to each observation point are calculated, thus determining the precise second position of that POI.

[0093] Step S240: Determine the pose offset matrix based on the second position of each point of interest.

[0094] Specifically, firstly, based on shape matching, multiple pairs of interest points are identified; the shape matching method can employ Procrustes analysis, which finds the best match by minimizing the sum of squared distances from one set of points to another, thereby determining the consistency between two shapes; or cluster analysis can be used to determine the corresponding pairs of interest points.

[0095] Subsequently, the pose offset matrix is ​​determined based on the second position of the interest point in each interest point pair and the third position of the corresponding interest point in the 3D model.

[0096] There are at least three pairs of interest points; the third position of each interest point in the 3D model is pre-configured known data.

[0097] In simple terms, the second position of each point of interest represents the position of the point of interest in the pelvis in reality, and the third position represents the position of the corresponding point of interest in the 3D model.

[0098] For each set of points, calculate their centroids (geometric centers). Let P... i Q represents the position of the i-th point of interest in the current context. i This indicates the third position in the 3D model. Therefore, the centroids are:

[0099]

[0100] Here, n is the number of points of interest.

[0101] Next, a covariance matrix H is constructed to determine the relationship between the two coordinate systems. This requires calculating the deviation vector of each point relative to its centroid, and then summing these deviations over all points. The formula is as follows:

[0102]

[0103] The covariance matrix H is decomposed using singular value decomposition (SVD). Let the result of the SVD decomposition be H = UΣVT, where U and V are orthogonal matrices, and Σ is a diagonal matrix.

[0104] The rotation matrix R can be obtained through the following equation:

[0105] R = VU T

[0106] To ensure that the rotation matrix is ​​orthogonal and its determinant is +1 (i.e., to ensure that it is a rigid transformation), if det(VUT) = -1, then the sign of the last column of V needs to be adjusted.

[0107] The translation vector t can be calculated using the following equation:

[0108] t = C P -RC Q

[0109] Finally, assemble the pose offset matrix T, which is a 4×4 matrix in the following form:

[0110]

[0111] The 0 here T This represents the transpose of the zero vector.

[0112] Step S250: Adjust the model pose based on the pose offset matrix to obtain the target pose.

[0113] The target pose represents the patient's current pelvic pose.

[0114] Specifically, the pose offset matrix displays the spatial difference between the patient's current pelvic position and the 3D model. Based on this pose offset matrix, the position and orientation of the 3D model can be adjusted to match the patient's actual condition. In other words, the 3D model is transformed using the pose offset matrix so that its final placement (target pose) perfectly matches the actual position and orientation of the patient's pelvis. The purpose of this is to ensure that the 3D model accurately reflects the patient's current anatomical structure, providing precise guidance for surgery.

[0115] This application provides a method for recognizing the pose of the human pelvis. The method includes: acquiring 2D data of the pelvis of a patient undergoing minimally invasive pelvic surgery and the pose of a pre-constructed 3D model of the patient; inputting the 2D pelvic data into a trained interest point tracking network for processing to obtain the first positions of multiple interest points; processing the first positions of each interest point based on triangulation to obtain the second positions of each interest point; determining a pose offset matrix based on the second positions of each interest point; and adjusting the model pose based on the pose offset matrix to obtain the target pose. The method proposed in this application can be understood as a multi-view approach in minimally invasive pelvic surgery navigation based on transfer learning. Figure 2 A D / 3D image registration method was developed, which designed a lightweight interest point tracking network based on U-Net and an attention mechanism. The introduced transfer learning module effectively solved the problem of insufficient data and improved the network's performance on real X-rays.

[0116] Corresponding to the above method, this application also provides a device for recognizing the posture of the human pelvis, such as... Figure 5 As shown, the device includes:

[0117] The acquisition unit 510 is used to acquire 2D data of the patient's pelvis and the model pose of a pre-constructed 3D model of the patient during minimally invasive pelvic surgery; wherein, the 2D data of the pelvis is data acquired from multiple perspectives of the patient's current pelvis via X-ray.

[0118] Processing unit 520 is used to input the pelvic 2D data into a trained interest point tracking network for processing to obtain the first position of multiple interest points; the first position represents the two-dimensional position of each interest point in the pelvic 2D data;

[0119] Furthermore, based on triangulation, the first position of each point of interest is processed to obtain the second position of each point of interest; the second position represents the three-dimensional position of each point of interest within the patient's body.

[0120] The determining unit 530 is used to determine the pose offset matrix based on the second position of each interest point;

[0121] The adjustment unit 540 is used to adjust the model pose based on the pose offset matrix to obtain a target pose, which represents the current pelvic pose of the patient.

[0122] The functions of each unit of the human pelvic posture recognition device provided in the above embodiments of this application can be realized through the above-described method steps. Therefore, the specific working process and beneficial effects of each unit in the human pelvic posture recognition device provided in the embodiments of this application will not be repeated here.

[0123] This application also provides an electronic device, such as... Figure 6 As shown, it includes a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640.

[0124] Memory 630 is used to store computer programs;

[0125] When the processor 610 executes the program stored in the memory 630, it performs the following steps:

[0126] Acquire 2D data of the pelvis of a patient during minimally invasive pelvic surgery and the model pose of a pre-constructed 3D model of the patient; wherein, the 2D data of the pelvis is data acquired from multiple perspectives of the patient's current pelvis via X-ray.

[0127] The 2D data of the pelvis is input into a trained interest point tracking network for processing to obtain the first position of multiple interest points; the first position represents the two-dimensional position of each interest point in the 2D data of the pelvis.

[0128] Based on triangulation, the first position of each point of interest is processed to obtain the second position of each point of interest; the second position represents the three-dimensional position of each point of interest in the patient's body.

[0129] Based on the second position of each point of interest, determine the pose offset matrix;

[0130] Based on the pose offset matrix, the model pose is adjusted to obtain the target pose, which represents the patient's current pelvic pose.

[0131] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0132] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0133] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0134] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0135] The implementation methods and beneficial effects of the various components of the electronic device in the above embodiments for solving the problem can be found in [reference needed]. Figure 2 The steps in the illustrated embodiments are used to implement the electronic device. Therefore, the specific working process and beneficial effects of the electronic device provided in this application will not be repeated here.

[0136] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform a method for recognizing the human pelvic posture as described in any of the above embodiments.

[0137] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the above embodiments of a method for recognizing human pelvic posture.

[0138] Those skilled in the art will understand that the embodiments in this application can be provided as methods, systems, or computer program products. Therefore, the embodiments in this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments in this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0140] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0141] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0142] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected," "coupled," or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0143] Although preferred embodiments have been described in this application, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the embodiments in this application are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments in this application.

[0144] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the embodiments of this application and their equivalents, then these modifications and variations are also intended to be included in the embodiments of this application.

Claims

1. A method for recognizing the position and posture of the human pelvis, characterized in that, A tracking and positioning module applied to a navigation system for minimally invasive pelvic surgery, the method comprising: Acquire 2D data of the pelvis of a patient during minimally invasive pelvic surgery and the model pose of a pre-constructed 3D model of the patient; wherein, the 2D data of the pelvis is data acquired from multiple perspectives of the patient's current pelvis via X-ray. The 2D pelvic data is input into a trained interest point tracking network for processing to obtain the first positions of multiple interest points. The first position represents the two-dimensional position of each interest point in the 2D pelvic data. The interest point tracking network includes an input layer, an encoder, a decoder, and an output layer connected in sequence. The encoder includes five convolutional blocks. Each convolutional block includes a first convolutional layer, a first batch normalization layer, a first activation function, a second convolutional layer, a second batch normalization layer, a second activation function, an attention mechanism unit, and a third activation function connected in sequence. The attention mechanism unit includes a channel attention module and a spatial attention module connected in sequence. The channel attention module includes parallel global average pooling submodules and global max pooling submodules, a first convolutional submodule and a second convolutional submodule connected in sequence to the global average pooling submodule, a third convolutional submodule and a fourth convolutional submodule connected in sequence to the global max pooling submodule, and a fourth activation function connected to the second convolutional submodule and the fourth convolutional submodule. The spatial attention module includes a channel pooling submodule, a fifth convolutional submodule, and a fifth activation function connected in sequence. Based on triangulation, the first position of each point of interest is processed to obtain the second position of each point of interest; the second position represents the three-dimensional position of each point of interest in the patient's body; wherein, the triangulation method includes: determining the baseline length based on the positions of two observation points for acquiring 2D pelvic data, and calculating the second position of each point of interest by combining the angle data at the time of acquisition and the first position of each point of interest through trigonometric functions; Based on the second position of each point of interest, determine the pose offset matrix; Based on the pose offset matrix, the model pose is adjusted to obtain the target pose, which represents the patient's current pelvic pose.

2. The method as described in claim 1, characterized in that, The training process of the interest point tracking network includes: The configured 3D data of the patient's pelvis is processed to obtain 2D data of the target pelvis from different angles; multiple points of interest are marked in the 3D data of the pelvis. Multiple target pelvic 2D data were used as training samples, and target pelvic 2D data carrying points of interest were used as training labels. Using the training samples and the training labels, the pre-trained interest point tracking network is trained to obtain a trained interest point tracking network.

3. The method as described in claim 2, characterized in that, The configured 3D data of the patient's pelvis was processed to obtain 2D data of the target pelvis from different angles, including: The 3D data of the pelvis is processed to obtain initial 2D data of the pelvis from different angles; For any initial pelvic 2D data from any angle, style transfer techniques are used to convert X-ray images into target pelvic 2D data with a consistent style, thereby reducing the style difference between X-ray and initial pelvic 2D data.

4. A device for recognizing the position and posture of the human pelvis, characterized in that, A tracking and positioning module for use in a navigation system for minimally invasive pelvic surgery, the device comprising: The acquisition unit is used to acquire 2D data of the patient's pelvis and the model pose of a pre-constructed 3D model of the patient during minimally invasive pelvic surgery; wherein, the 2D data of the pelvis is data acquired from multiple perspectives of the patient's current pelvis via X-ray. A processing unit is used to input the pelvic 2D data into a trained interest point tracking network for processing to obtain the first positions of multiple interest points; the first position represents the two-dimensional position of each interest point in the pelvic 2D data; the interest point tracking network includes: an input layer, an encoder, a decoder, and an output layer connected in sequence; wherein, the encoder includes: 5 convolutional blocks; each convolutional block includes a first convolutional layer, a first batch normalization layer, a first activation function, a second convolutional layer, a second batch normalization layer, a second activation function, an attention mechanism unit, and a third activation function connected in sequence; the attention mechanism unit includes a channel attention module and a spatial attention module connected in sequence; the channel attention module includes a parallel global average pooling submodule and a global max pooling submodule, a first convolutional submodule and a second convolutional submodule connected in sequence to the global average pooling submodule, a third convolutional submodule and a fourth convolutional submodule connected in sequence to the global max pooling submodule, and a fourth activation function connected to the second convolutional submodule and the fourth convolutional submodule; the spatial attention module includes a channel pooling submodule, a fifth convolutional submodule, and a fifth activation function connected in sequence; Furthermore, based on triangulation, the first position of each point of interest is processed to obtain the second position of each point of interest; the second position represents the three-dimensional position of each point of interest within the patient's body; wherein, the triangulation includes: determining the baseline length based on the positions of two observation points for acquiring 2D pelvic data, and calculating the second position of each point of interest using trigonometric functions in combination with the angle data acquired and the first position of each point of interest. The determining unit is used to determine the pose offset matrix based on the second position of each interest point; An adjustment unit is used to adjust the model pose based on the pose offset matrix to obtain a target pose, which represents the patient's current pelvic pose.

5. An electronic device, characterized in that, The electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method according to any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-3.

Images