Image registration method, device, storage medium and electronic device

Abstract

The present application provides an image registration method, device, storage medium and electronic device, which is applied to the field of medical image processing technology. The method includes: obtaining three-dimensional imaging data of a target person, obtaining a two-dimensional scanned image of the target person, and a machine feedback posture corresponding to the two-dimensional scanned image, using a posture calculation model, obtaining a predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture, and performing registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture. The method can use the machine feedback posture as the initial posture and calculate the predicted posture in combination with the posture calculation model, so that the result of the registration algorithm approaches the global optimal solution, can avoid falling into the local optimal solution, and improves the registration efficiency and registration accuracy.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and more specifically, to an image registration method, device, storage medium and electronic device in the field of medical image processing technology. Background Art

[0002] During surgery, accurate positioning of the surgical position is very important for the success and effect of the operation. Therefore, in order to accurately locate the surgical position and improve surgical accuracy, the existing technology can combine two-dimensional X-ray images and three-dimensional cone-beam computed tomography images for surgical navigation and positioning of the surgical position. However, only aligned X-ray images and three-dimensional cone-beam computed tomography images can achieve accurate positioning. If three-dimensional cone-beam computed tomography images are collected on the human body during surgery, radiation leakage will result. If a gradient-based iterative optimization algorithm is used to achieve alignment, it is necessary to find the optimal solution through multiple iterations, which will consume a lot of computing power and a long calculation time. It is necessary to provide a method for quickly aligning two-dimensional X-ray images and three-dimensional cone-beam computed tomography images. Summary of the Invention

[0003] The embodiments of the present application provide an image registration method, device, storage medium and electronic device. The method can obtain a predicted pose by using the machine feedback pose as the initial pose and combining it with a pose calculation model, so that the result of the registration algorithm approaches the global optimal solution, avoids falling into the local optimal solution, and improves the registration efficiency and accuracy.

[0004] In a first aspect, an embodiment of the present application provides an image registration method, the method comprising:

[0005] Acquire three-dimensional imaging data of the target person;

[0006] Acquire a two-dimensional scanned image of the target person and a machine feedback posture corresponding to the two-dimensional scanned image;

[0007] Using a posture calculation model, based on the two-dimensional scanned image and the machine feedback posture, a predicted posture corresponding to the two-dimensional scanned image is obtained;

[0008] The three-dimensional imaging data and the two-dimensional scanned image are registered based on the predicted posture.

[0009] In a second aspect, an embodiment of the present application provides an image registration device, the device comprising:

[0010] A three-dimensional data acquisition unit, used to acquire three-dimensional imaging data of a target person;

[0011] A two-dimensional data acquisition unit, configured to acquire a two-dimensional scanned image of the target person and a machine feedback posture corresponding to the two-dimensional scanned image;

[0012] A posture prediction unit, configured to adopt a posture calculation model to obtain a predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture;

[0013] A registration processing unit is used to perform registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture.

[0014] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0016] In one or more embodiments of the present application, three-dimensional imaging data of a target person is obtained, a two-dimensional scanned image of the target person is obtained, and a machine feedback posture corresponding to the two-dimensional scanned image is obtained. A posture calculation model is used to obtain a predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture. The three-dimensional imaging data and the two-dimensional scanned image are then registered based on the predicted posture. By using the machine feedback posture as the initial posture and combining it with the posture calculation model to calculate the predicted posture, the result of the registration algorithm approaches the global optimal solution, avoiding falling into the local optimal solution, thereby improving the registration efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 This is a flowchart of an image registration method provided in an embodiment of the present application;

[0019] Figure 2 This is a flow chart of a posture calculation model training method provided in an embodiment of the present application;

[0020] Figure 3This is a flowchart of a loss value calculation method provided in an embodiment of the present application;

[0021] Figure 4 This is a flowchart of a loss value calculation method provided in an embodiment of the present application;

[0022] Figure 5 This is an example diagram of model training and loss value calculation provided in an embodiment of the present application;

[0023] Figure 6 This is a flow chart of an image preprocessing method provided in an embodiment of the present application;

[0024] Figure 7 This is an example schematic diagram of a two-dimensional scan image preprocessing provided by an embodiment of the present application;

[0025] Figure 8 This is a flow chart of a three-dimensional data preprocessing method provided in an embodiment of the present application;

[0026] Figure 9 This is a flow chart of an image preprocessing method provided in an embodiment of the present application;

[0027] Figure 10 This is a schematic diagram illustrating an example of an image registration effect provided in an embodiment of the present application;

[0028] Figure 11 is a structural diagram of an image registration device provided in an embodiment of the present application;

[0029] Figure 12 is a structural diagram of an image registration device provided in an embodiment of the present application;

[0030] Figure 13 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0032] To allow the surgeon to visualize the surgical instruments and surgical location, three-dimensional imaging data and two-dimensional scanned images can be combined for surgical guidance, improving surgical safety and precision, minimizing patient trauma, and reducing the risk of complications. The three-dimensional imaging data can be three-dimensional cone-beam computed tomography (CBCT) data, and the two-dimensional scanned images can be two-dimensional X-ray images. CBCT is a medical imaging technique that uses cone-beam X-rays to scan an object and reconstruct a three-dimensional image using a computer. For example, an X-ray generator can perform a circular digital projection around the object at a low radiation dose. The X-ray source and detector rotate around the object, typically 360 degrees, to obtain two-dimensional projections from multiple angles. These projections are then collected and reconstructed by a computer to ultimately generate a three-dimensional image. X-ray imaging is an imaging technique that uses X-rays to form images after they penetrate an object, revealing its internal structure. When X-rays penetrate an object, tissues of different densities and atomic numbers, such as bone, muscle, and fat, absorb the X-rays to varying degrees. High-density tissue, such as bone, absorbs more X-rays, while low-density tissue, such as air, absorbs less. The X-rays that penetrate the body are received by the detector, forming a grayscale image. The brightness of the image reflects the differences in X-ray absorption by the tissues, thereby revealing the internal structure of the object. Accurately locating the surgical site requires registering the 3D imaging data with the 2D scan image based on pose. Pose refers to information such as the position, angle, and focal length of the X-ray source when the 2D scan image was captured.

[0033] It is understandable that as the operation progresses and the surgical position changes, the doctor can collect two-dimensional scanning images of the surgeon multiple times to obtain real-time and accurate surgical positions. The postures of these two-dimensional scanning images may be different, but two-dimensional scanning images have certain limitations in the accuracy of displaying blood vessels and tissue structures. They can only provide anatomical images from a single perspective and cannot show the true structure of human tissues and organs in three-dimensional space. Although three-dimensional imaging data is more accurate, it involves high-dose X-rays. If it is used frequently during surgery, collecting three-dimensional imaging data for two-dimensional scanning images of different postures and their registration will cause patients and medical staff to be exposed to higher radiation.

[0034] The present application provides an image registration device that can collect three-dimensional imaging data of a target person before surgery, collect real-time two-dimensional scanned images of the target person during surgery, place the three-dimensional imaging data and the two-dimensional scanned images in the same coordinate system, and move and transform the three-dimensional imaging data based on the position and posture of the two-dimensional scanned images, thereby achieving registration of the three-dimensional imaging data and the two-dimensional scanned images. The image registration method provided in the present application can be implemented by a computer program and can be run on an image registration device based on the von Neumann system. The computer program can be integrated into an application or run as a standalone tool application.

[0035] The image registration method provided in this application is described in detail below with reference to specific embodiments.

[0036] See Figure 1 , provides a flow chart of an image registration method according to an embodiment of the present application. Figure 1 As shown, the method of the embodiment of the present application may include the following steps S101-S104.

[0037] S101: Acquire three-dimensional imaging data of a target person.

[0038] Specifically, since the collection of three-dimensional imaging data involves high-dose X-rays, three-dimensional imaging data can be collected in advance for the target person before the operation. The image registration device can directly obtain the pre-collected three-dimensional imaging data for the target person, where the target person can be a patient who needs to undergo surgery.

[0039] S102, obtaining a two-dimensional scan image of a target person and a machine feedback posture corresponding to the two-dimensional scan image.

[0040] Specifically, as the surgical process progresses and the surgical location and medical device positions change, doctors and other personnel can use a two-dimensional image acquisition machine to acquire a two-dimensional scan image of the target person in order to obtain real-time surgical conditions. The two-dimensional image acquisition machine can be used to acquire two-dimensional scan images such as X-ray images. For example, the two-dimensional image acquisition machine can be an X-ray machine. After acquiring the two-dimensional scan image, doctors and other relevant personnel can send a registration instruction to the image registration device to instruct the image registration device to align the acquired two-dimensional scan image with the pre-acquired three-dimensional imaging data. The image registration device can obtain a two-dimensional scan image of the target person and a machine-feedback posture feedbacked by the two-dimensional image acquisition machine. The machine-feedback posture is the posture corresponding to the acquisition of the two-dimensional scan image as feedbacked by the two-dimensional image acquisition machine. It can include information such as the angle of the X-ray light source, the position of the X-ray light source, the focal length of the machine, the distance between the target person and the X-ray light source, and the distance between the target person and the detection plate.

[0041] Optionally, in order to improve the efficiency and accuracy of the registration process, the image registration device can establish a unified coordinate system to facilitate the alignment of two-dimensional scanning images and three-dimensional imaging data obtained by different devices to the same reference frame. For example, the unified coordinate system can be a world coordinate system, which is a fixed three-dimensional coordinate system that can be used to define the absolute position of an object in space. The origin of the unified coordinate system can be set by the image registration device or relevant medical personnel, for example, it can be the center of the target person's body. The posture of all devices can be described relative to the unified coordinate system, so the machine feedback posture can be the posture information in the unified coordinate system. For example, the position of the X-ray light source can be the coordinate in the unified coordinate system, and the angle of the X-ray light source can be the angle in the unified coordinate system.

[0042] S103 , using a posture calculation model, obtaining a predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the posture fed back by the machine.

[0043] Specifically, the machine feedback posture actually has certain errors and is not the accurate posture of the two-dimensional scanned image. If the machine feedback posture is directly used for registration processing, the registration accuracy will be reduced, making it difficult to determine the accurate surgical position. However, if the corresponding posture is directly obtained based on the two-dimensional scanned image, it will not only require a large amount of calculation and calculation time, thereby delaying the surgical process, but may also fall into a local optimal solution and fail to obtain a global optimal solution or an accurate posture. The image registration device can train a posture calculation model, and the posture calculation model can use the machine feedback posture as the initial posture to predict the posture corresponding to the two-dimensional scanned image. Therefore, the image registration device can use the posture calculation model to obtain the predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture. The predicted posture is the posture output by the posture calculation model for the two-dimensional scanned image, which is the actual posture of the two-dimensional scanned image predicted by the posture calculation model.

[0044] Optionally, the predicted pose may be pose information in a unified coordinate system.

[0045] S104: performing registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture.

[0046] Specifically, the image registration device can also place the three-dimensional imaging data in a unified coordinate system, and then perform registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture, so as to facilitate doctors and others to use the registered three-dimensional imaging data and two-dimensional scanned images to perform surgery on the target person. Among them, the registration processing can be to move the three-dimensional imaging data to a position that matches the predicted position of the two-dimensional scanned image, and then use the forward projection algorithm (Forward Projection Algorithm) to project the three-dimensional imaging data to obtain the registered image. Doctors and others can use the registered image to guide surgery.

[0047] Optionally, the image registration device may place the three-dimensional imaging data in a unified coordinate system, and obtain the coordinates of each voxel in the three-dimensional imaging data in the unified coordinate system.

[0048] In an embodiment of the present application, three-dimensional imaging data of a target person is obtained, along with a two-dimensional scanned image of the target person and a machine feedback pose corresponding to the two-dimensional scanned image. A pose calculation model is used to obtain a predicted pose corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback pose. Registration processing is then performed on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted pose. By using the machine feedback pose as the initial pose and combining it with the pose calculation model to calculate the predicted pose, the result of the registration algorithm approaches the global optimal solution, avoiding falling into a local optimal solution, thereby improving registration efficiency and accuracy.

[0049] See Figure 2 , which is a flowchart of a posture calculation model training method according to an embodiment of the present application, may further include the following steps before step S103:

[0050] S201, creating an initial pose calculation model, obtaining a sample two-dimensional scan image, and a sample machine feedback pose and a sample real pose corresponding to the sample two-dimensional scan image.

[0051] Specifically, the image registration device can create an initial pose calculation model, which can be a deep residual neural network (Residual Network 50, ResNet50). The initial pose calculation model can have the ability to predict the pose offset data between the pose corresponding to the two-dimensional scanned image and the machine feedback pose based on the machine feedback pose and the two-dimensional scanned image. The pose offset data can be the difference between the pose corresponding to the two-dimensional scanned image and the machine feedback pose. In order to further optimize the ability of the initial pose calculation model to obtain pose offset data, the image registration device can obtain sample two-dimensional scanned images, as well as sample machine feedback poses and sample true poses corresponding to the sample two-dimensional scanned images, to train the initial pose calculation model.

[0052] Optionally, the sample two-dimensional scanning image obtained by the image registration device can be a two-dimensional scanning image obtained by doctors and other relevant staff based on a real human body. The sample machine feedback posture is the posture corresponding to the sample two-dimensional scanning image when it is collected, which is fed back by the two-dimensional image acquisition machine. The sample real posture can be the accurate posture matched by the image registration device according to the range of all activities of the two-dimensional image acquisition machine. Therefore, the calculation of the sample real posture requires a lot of computing power but has high accuracy.

[0053] Optionally, the sample two-dimensional scanned image may also be a digitally reconstructed radiograph (DRR). DRR generates a two-dimensional image similar to a traditional two-dimensional scanned image from digitized three-dimensional imaging data by computer simulation of the attenuation and exposure process of X-rays. The image registration device can obtain the sample three-dimensional imaging data and the sample's true pose, wherein the sample three-dimensional imaging data may be input by a doctor or other relevant staff member, and the sample's true pose may be multiple poses obtained by the image registration device within the range of all activities of the two-dimensional image acquisition machine. The image registration device can input the sample three-dimensional imaging data and the sample's true pose into a differentiable digitally reconstructed radiograph (DiffDRR). DiffDRR is a PyTorch-based DRR generator that can generate a corresponding sample two-dimensional scanned image based on the sample's true pose and the sample three-dimensional imaging data.

[0054] S202: Input the sample two-dimensional scan image and the sample machine feedback posture into the initial posture calculation model to obtain the training posture offset data output by the initial posture calculation model.

[0055] Specifically, the initial pose calculation model can output pose offset data based on the 2D scan image and the machine feedback pose. The pose offset data is the difference between the actual pose of the 2D scan image and the machine feedback pose predicted by the initial pose calculation model. Therefore, the image registration device can input the sample 2D scan image and the sample machine feedback pose into the initial pose calculation model to obtain training pose offset data output by the initial pose calculation model. The training pose offset data is the difference between the actual pose of the sample 2D scan image and the sample machine feedback pose predicted by the initial pose calculation model.

[0056] S203: Based on the training pose offset data and the sample machine feedback pose, obtain the training predicted pose corresponding to the sample two-dimensional scanned image.

[0057] Specifically, the image registration device can obtain the training predicted pose corresponding to the sample two-dimensional scanned image based on the training pose offset data and the sample machine feedback pose. The training predicted pose is the true pose of the two-dimensional scanned image predicted by the initial pose calculation model. The image registration device can sum the training pose offset data and the sample machine feedback pose, and the obtained sum is the training predicted pose.

[0058] S204: Based on the training predicted pose and the actual pose of the sample, the parameters of the initial pose calculation model are adjusted until the model training is completed to obtain the pose calculation model.

[0059] Specifically, the image registration device can perform model training on the initial pose calculation model based on the training predicted pose and the sample true pose, and adjust the parameters of the initial pose calculation model during the model training process to reduce the loss value between the training predicted pose and the sample true pose, so that the training predicted pose is closer to the sample true pose, thereby improving the accuracy of the initial pose calculation model, until the model training is completed and the pose calculation model is obtained.

[0060] Optionally, the image registration device performs model training on the initial pose calculation model until the initial pose calculation model meets the training termination condition, thereby completing the model training and obtaining the pose calculation model. The training termination condition is used to determine whether the initial pose calculation model has the ability to obtain accurate pose offset data. The training termination condition can be an initial setting of the image registration device or can be set by the user or relevant staff.

[0061] Optionally, the training termination condition may be that the model training rounds for the initial pose calculation model have reached a preset round. For example, the preset rounds may be 10,000 times. Using the training rounds as the training termination condition is intuitive and easy to understand, and does not require complex calculations or judgment conditions. It is only necessary to set the rounds at the beginning of training, making the management of the training process simpler.

[0062] Optionally, the training termination condition may include the loss value falling below a preset loss value and maintaining it for a preset time. The preset loss value is used to determine whether the prediction accuracy of the initial posture calculation model has reached the standard, and the preset time is used to determine whether the stability of the initial posture calculation model has reached the standard. The preset loss value and the preset time can be the initial settings of the image registration device, and can also be set by the user or relevant staff. For example, the preset loss value can be 5%, and the preset time can be 10 rounds of model training.

[0063] In an embodiment of the present application, an initial posture calculation model is created, a sample two-dimensional scanned image, and a sample machine feedback posture and a sample true posture corresponding to the sample two-dimensional scanned image are obtained, the sample two-dimensional scanned image and the sample machine feedback posture are input into the initial posture calculation model, and the training posture offset data output by the initial posture calculation model is obtained. Based on the training posture offset data and the sample machine feedback posture, the training predicted posture corresponding to the sample two-dimensional scanned image is obtained. Based on the training predicted posture and the sample true posture, the initial posture calculation model is parameter-adjusted until the model training is completed, thereby obtaining a posture calculation model. By using the sample two-dimensional scanned image and the sample machine feedback posture to train the initial posture calculation model, a posture calculation model is obtained, and the posture calculation model is used to obtain the predicted posture of the two-dimensional scanned image, thereby improving the efficiency and accuracy of the predicted posture calculation.

[0064] In addition to obtaining the loss value between the training predicted pose and the sample's true pose, the image registration device can make the training predicted pose obtained during the model training process closer to the sample's true pose. In order to further improve the accuracy of the pose calculation model, the two-dimensional scanned image corresponding to the training predicted pose can also be simulated. If the two-dimensional scanned image corresponding to the training predicted pose is closer to the sample's true pose, it means that the training predicted pose is closer to the sample's true pose.

[0065] See Figure 3 , provides a flowchart of a loss value calculation method according to an embodiment of the present application, in which step S204 may include the following steps:

[0066] S301, generating a simulated two-dimensional scanned image based on the training predicted pose.

[0067] Specifically, the image registration device can input the training predicted pose into DiffDRR. DiffDRR can generate a simulated 2D scan image for the sample 3D imaging data based on the training predicted pose. The difference between the simulated 2D scan image and the sample 2D scan image can also represent the difference between the training predicted pose and the sample's true pose, which can be used to measure the prediction accuracy of the initial pose calculation model. Therefore, the image registration device can adjust the parameters of the initial pose calculation model based on the training predicted pose and the sample's true pose, as well as the simulated 2D scan image and the sample 2D scan image, until the model training is completed, thereby obtaining the pose calculation model.

[0068] S302: Obtain a first loss value based on the training predicted pose and the true pose of the sample.

[0069] Specifically, the image registration device can obtain a first loss value based on the training predicted pose and the sample true pose. The first loss value can be the distance (Distance) between the training predicted pose and the sample true pose. For example, if the pre-trained predicted pose and the sample true pose both have multiple degrees of freedom, the pre-trained predicted pose and the sample true pose can be placed in a high-level space to calculate the distance between the point corresponding to the pre-trained predicted pose and the point corresponding to the sample true pose.

[0070] Optionally, the first loss value may be composed of a geodesic loss and a dual geodesic loss, as follows:

[0071]

[0072] in, is the first loss value, can be the geodesic loss, It can be a dual geodesic loss, which is usually used to measure the difference between two rotation matrices. Using geodesic loss as the loss function, that is, using the geodesic distance between the training predicted pose and the sample's true pose as the regression loss, you can give two rotation matrices , in three-dimensional space, rotations can be obtained through the special orthogonal group To express it, the geodesic distance is a natural distance metric on this group, which reflects the shortest path between two rotations. The angular distance between two rotation matrices is calculated as follows:

[0073]

[0074]

[0075] in, is The logarithmic mapping on The logarithmic mapping on the pose can be obtained The geodesic loss on , is as follows:

[0076]

[0077] The dual geodesic loss is an extension of the geodesic loss, which takes into account the difference between rotation and translation. In three-dimensional space, a complete posture can be described by a rotation matrix and a translation vector, and is described by a special Euclidean group. To indicate that, Assuming the geodesic distance with length units above, based on the focal length f of the two-dimensional scanned image, the above angular distance can be converted into arc length using the following formula:

[0078]

[0079] The dual geodesic loss combines the geodesic distance of rotation and the Euclidean distance between two translation vectors, so it is similar to the Euclidean distance of translation. After combining, we can obtain The dual geodesic loss on is given by:

[0080]

[0081] S303: Obtain a second loss value based on the simulated two-dimensional scanning image and the sample two-dimensional scanning image.

[0082] Specifically, the image registration device can obtain a second loss value based on the 2D scanned image and the sample 2D scanned image. The second loss value can be a multi-scale normalized cross-correlation (mNCC). By calculating normalized cross-correlation at multiple scales, it can capture image features at different scales, reduce errors that may be introduced by single-scale analysis, and improve the robustness and reliability of the algorithm. mNCC can compare the similarity between two images. Its value range is [-1, 1]. The closer the mNCC value is to 1, the more similar the two images are.

[0083] S304, obtaining a summary loss value based on the first loss value and the second loss value, and performing parameter adjustment processing on the initial posture calculation model based on the summary loss value until the model training is completed to obtain a posture calculation model.

[0084] Specifically, the image registration device can combine the first loss value and the second loss value to obtain a summary loss value, and adjust the parameters of the initial pose calculation model during the model training process to reduce the summary loss value, thereby improving the accuracy of the initial pose calculation model until the model training is completed and the pose calculation model is obtained.

[0085] Optionally, the image registration device can calculate the difference between 1 and the second loss value, and the product of the predicted ratio parameter and the first loss value, and then calculate the sum of the difference between 1 and the second loss value and the product of the predicted ratio parameter and the first loss value to obtain a summary loss value, where the predicted ratio parameter can be the initial setting of the image registration device, or can be set by relevant staff such as doctors, for example, it can be 0.01.

[0086] In an embodiment of the present application, a simulated two-dimensional scanned image is generated based on the training predicted pose, a first loss value is obtained based on the training predicted pose and the sample true pose, a second loss value is obtained based on the simulated two-dimensional scanned image and the sample two-dimensional scanned image, a summary loss value is obtained based on the first loss value and the second loss value, and the parameters of the initial pose calculation model are adjusted based on the summary loss value until the model training is completed to obtain the pose calculation model. By combining the first loss value and the second loss value, the initial pose calculation model can learn and optimize the pose prediction ability from multiple perspectives, reduce the model's overfitting of the training data, and improve generalization ability.

[0087] Although the machine feedback pose fed back by the two-dimensional image acquisition machine is not accurate, it is relatively close to the real pose corresponding to the two-dimensional scanned image. In order to avoid the pose calculation model from falling into the local optimal solution, a constraint mechanism can be introduced in the model training stage. During the model training process, the difference between the training predicted pose predicted by the initial pose calculation model and the real pose of the sample is calculated, so as to constrain the predicted pose to a certain range centered on the machine feedback pose.

[0088] See Figure 4 , provides a flowchart of a loss value calculation method according to an embodiment of the present application, in which step S304 may include the following steps:

[0089] S401, obtaining a third loss value based on the sample machine feedback pose and the training predicted pose.

[0090] Specifically, the image registration device can obtain a third loss value based on the sample machine feedback pose and the training predicted pose. The third loss value can be the absolute value of the difference between the sample machine feedback pose and the training predicted pose, or the absolute value of the training offset data.

[0091] S402, obtaining a summary loss value based on the first loss value, the second loss value, and the third loss value, and performing parameter adjustment processing on the initial posture calculation model based on the summary loss value until the model training is completed to obtain a posture calculation model.

[0092] Specifically, the image registration device can combine the first loss value, the second loss value and the third loss value to obtain a summary loss value, and adjust the parameters of the initial pose calculation model during the model training process to reduce the summary loss value, thereby improving the accuracy of the initial pose calculation model until the model training is completed and the pose calculation model is obtained.

[0093] Optionally, the summary loss value is calculated as follows:

[0094]

[0095] in, is the summary loss value, is the second loss value, is the first loss value, is the third loss value.

[0096] Please also see Figure 5 , an example schematic diagram of model training and loss value calculation is provided for an embodiment of the present application. The image registration device can input the sample's true pose into DiffDRR for simulation to obtain a sample two-dimensional scanned image, and input the sample two-dimensional scanned image and its corresponding sample machine feedback pose into the initial pose calculation model to obtain the training pose offset data output by the initial pose calculation model. The training pose offset data and the sample machine feedback pose can be combined to obtain the training predicted pose. The training predicted pose is input into DiffDRR for simulation to obtain a simulated two-dimensional scanned image. Then, a first loss value can be obtained based on the sample's true pose and the training predicted pose, a second loss value can be obtained based on the sample two-dimensional scanned image and the simulated two-dimensional scanned image, and a third loss value can be obtained based on the training offset data. Then, a summary loss value can be obtained based on the first loss value, the second loss value, and the third loss value.

[0097] In an embodiment of the present application, a third loss value is obtained based on the sample machine feedback pose and the training predicted pose, a summary loss value is obtained based on the first loss value, the second loss value, and the third loss value, and the parameters of the initial pose calculation model are adjusted based on the summary loss value until the model training is completed, thereby obtaining the pose calculation model. By applying a constraint mechanism during the model training process, the model output is kept within a certain range centered on the machine feedback pose, thereby avoiding falling into a local optimal solution and further improving the accuracy of the predicted pose.

[0098] In order to further improve the accuracy of the pose calculation model, the two-dimensional scanned image can be preprocessed. During the preprocessing, the key features of the image can be enhanced, noise can be removed, and the format and scale of the image can be standardized, so that the pose calculation model can efficiently learn the two-dimensional scanned image, thereby improving the reliability and stability of the predicted pose.

[0099] See Figure 6 , which is a flowchart of an image preprocessing method according to an embodiment of the present application, may further include the following steps before step S102:

[0100] S501: Acquire an initial two-dimensional scan image of a target person.

[0101] Specifically, the image registration device may obtain an initial two-dimensional scan image of the target person, where the initial two-dimensional scan image is a two-dimensional scan image captured by a two-dimensional image capture machine.

[0102] S502 , performing normalization and inversion processing on the initial two-dimensional scanned image to obtain a two-dimensional scanned image.

[0103] Specifically, the image registration device can perform normalization processing on the initial two-dimensional scanned image, thereby normalizing the pixel values ​​in the initial two-dimensional scanned image to between 0 and 1. This can eliminate the problem of inconsistent pixel value ranges between different images due to factors such as lighting conditions and imaging equipment differences. This helps to highlight key features in the image, enabling the pose calculation model to more accurately identify and learn these features. The grayscale values ​​of the normalized initial two-dimensional scanned image can then be inverted to obtain a two-dimensional scanned image. The inversion process can enhance the contrast of the image, making dark features more prominent and highlighting dark details.

[0104] Optionally, the image registration device can also perform contrast limited adaptive histogram equalization on the initial two-dimensional scanned image after the inversion process to obtain a two-dimensional scanned image. Figure 7 , an example schematic diagram of a two-dimensional scanned image preprocessing is provided for an embodiment of the present application. The image registration device can perform normalization processing, inversion processing, and contrast-limited adaptive histogram equalization and other preprocessing on the initial two-dimensional scanned image to obtain a two-dimensional scanned image.

[0105] In the embodiment of the present application, an initial 2D scan image of the target person is obtained, and normalized and inverted to obtain a 2D scan image. The normalization and inversion processes highlight key features and dark details in the 2D scan image, thereby improving the generalization capability of the pose calculation model, reducing the risk of overfitting, and increasing the computational efficiency and accuracy of pose prediction.

[0106] In addition to preprocessing two-dimensional scanned images, the image registration device can also preprocess three-dimensional imaging data, thereby improving registration accuracy and image clarity.

[0107] See Figure 8 , which provides a flow chart of a three-dimensional data preprocessing method according to an embodiment of the present application. In one or more embodiments of the present application, step S101 may include the following steps:

[0108] S601: Acquire initial three-dimensional imaging data of a target person.

[0109] Specifically, the initial three-dimensional imaging data may be three-dimensional imaging data collected in advance for the target person before the operation.

[0110] S602 , performing weight-increasing processing on the skeleton parts in the initial three-dimensional imaging data, and performing weight-reducing processing on the non-skeletal parts in the initial three-dimensional imaging data, to obtain three-dimensional imaging data of the target person.

[0111] Specifically, the Hounsfield Unit (HU) value, a quantitative unit used to describe the X-ray absorption rate of a substance, can be used to distinguish tissues of different densities. The image registration device can obtain the HU value of each voxel in the initial 3D imaging data to determine the skeletal and non-skeletal parts in the initial 3D imaging data. The non-skeletal parts can be air and soft tissue. The skeletal parts are then weighted up and the non-skeletal parts are weighted down to obtain the 3D imaging data. This reduces the interference of the non-skeletal parts on the skeletal structure, enhances the contrast and clarity of the skeletal structure, and highlights the details of the skeletal structure. This improves the visualization of the 3D imaging data while also increasing the efficiency and accuracy of image registration.

[0112] Optionally, the HU value can reflect the density and composition of the tissue. Different tissues have different HU value ranges, so the image registration device can determine the bone parts and non-bone parts in the initial three-dimensional imaging data based on the preset HU value. The parts with HU values ​​greater than the preset HU value can be determined as bone parts, and the parts with HU values ​​less than the preset HU value can be determined as non-bone parts. The preset HU value can be the initial setting of the image registration device, and can also be set by the user or relevant medical staff. Since the HU value of air is approximately -1000, indicating that air absorbs the least X-rays, the HU value of soft tissue can be between -100 and +100, such as muscle, fat, etc. The HU value of bone is higher, usually above +1000, such as cortical bone, so the preset HU value can be 1000.

[0113] In an embodiment of the present application, initial 3D imaging data of a target person is obtained, and the weights of the skeletal parts in the initial 3D imaging data are increased, while the weights of the non-skeletal parts in the initial 3D imaging data are decreased, thereby obtaining 3D imaging data of the target person. By enhancing the contrast and clarity of the skeletal structure, the details of the skeletal structure are highlighted, improving the visualization of the 3D imaging data while also enhancing the efficiency and accuracy of image registration.

[0114] Since the posture calculation model can output posture offset data based on the two-dimensional scan image and the machine feedback posture, rather than directly outputting the predicted posture, the image registration device needs to perform calculations based on the data output by the posture calculation model.

[0115] See Figure 9, which provides a flowchart of an image preprocessing method according to an embodiment of the present application, wherein step S103 may include the following steps:

[0116] S701: Input the two-dimensional scan image and the machine feedback posture into the posture calculation model to obtain the posture offset data output by the posture calculation model.

[0117] Specifically, the image registration device can input the two-dimensional scanned image and the machine feedback posture into the posture calculation model, and then obtain the posture offset data output by the posture calculation model. The posture offset data is the difference between the actual posture of the two-dimensional scanned image predicted by the posture calculation model and the machine feedback posture.

[0118] S702: Based on the posture offset data and the machine feedback posture, a predicted posture corresponding to the two-dimensional scanned image is obtained.

[0119] Specifically, the image registration device can calculate the predicted pose corresponding to the two-dimensional scanned image based on the pose offset data and the machine feedback pose. For example, the sum of the pose offset data and the machine feedback pose can be calculated to obtain the predicted pose. Please also refer to Figure 10 , an example schematic diagram of an image registration effect is provided for an embodiment of the present application. A checkerboard pattern is a visualization method for intuitively displaying the registration effect. The degree of alignment between the two images can be displayed alternately in the form of a checkerboard. The three-dimensional imaging data can be projected using a front projection algorithm, and a checkerboard pattern generated by the projection and the two-dimensional scanned image can be obtained, that is, the checkerboard pattern before image registration. It can be observed in the checkerboard pattern before image registration that there is an obvious deviation between the projection of the three-dimensional imaging data and the two-dimensional scanned image, while the projection of the three-dimensional imaging data in the checkerboard pattern after image registration has overlapped with the two-dimensional scanned image.

[0120] In an embodiment of the present application, a 2D scanned image and machine-feedback pose are input into a pose calculation model. The pose calculation model then outputs pose offset data, which is then used to calculate the predicted pose corresponding to the 2D scanned image. By inputting the 2D scanned image and machine-feedback pose into the pose calculation model and using the pose offset data output by the model to calculate the predicted pose, the accuracy and robustness of the predicted pose are improved.

[0121] In one or more embodiments of the present application, step S104 may include:

[0122] The three-dimensional imaging data is rigidly transformed based on the predicted pose to align the three-dimensional imaging data with the two-dimensional scanned image.

[0123] Specifically, the image registration device can perform a rigid transformation on the 3D imaging data in a unified coordinate system based on the predicted pose, thereby moving the 3D imaging data to a position that matches the predicted pose and is aligned with the 2D scanned image. Rigid transformation is a geometric transformation that maintains the shape and size of the 3D imaging data while only changing its position and orientation. This means that rigid transformation does not change the distance between any two points within the 3D imaging data, nor does it alter the shape or volume of the 3D imaging data. Rigid transformation consists of translation and rotation, and may also include reflection.

[0124] In the embodiment of the present application, the structural integrity of the three-dimensional imaging data is maintained through rigid transformation, error accumulation is reduced, the registration process is simplified, and the overall performance and reliability of image registration are improved.

[0125] The following will be combined with the Figure 11 -Attached Figure 12 , an image registration device provided in an embodiment of the present application is introduced in detail. Figure 11 -Attached Figure 12 An image registration device in the present application Figures 1-10 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figures 1-10 The embodiment shown.

[0126] See Figure 11 , which shows a schematic diagram of the structure of an image registration device provided by an exemplary embodiment of the present application. This image registration device can be implemented as all or part of a device through software, hardware, or a combination of both. The device 1 includes a 3D data acquisition unit 11, a 2D data acquisition unit 12, a pose prediction unit 13, and a registration processing unit 14.

[0127] A three-dimensional data acquisition unit 11 is used to acquire three-dimensional imaging data of a target person;

[0128] A two-dimensional data acquisition unit 12 is used to acquire a two-dimensional scan image of the target person and a machine feedback posture corresponding to the two-dimensional scan image;

[0129] A posture prediction unit 13 is configured to use a posture calculation model to obtain a predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture;

[0130] The registration processing unit 14 is configured to perform registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture.

[0131] In this embodiment, three-dimensional imaging data of a target person is obtained, along with a two-dimensional scanned image of the target person and a machine-feedback pose corresponding to the two-dimensional scanned image. A pose calculation model is then used to obtain a predicted pose corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine-feedback pose. Registration processing is then performed on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted pose. By using the machine-feedback pose as the initial pose and combining it with the pose calculation model to calculate the predicted pose, the results of the registration algorithm approach the global optimal solution, avoiding falling into a local optimal solution and improving registration efficiency and accuracy.

[0132] See Figure 12 , which shows a schematic diagram of the structure of an image registration device provided by an exemplary embodiment of the present application. The image registration device can be implemented as all or part of the device through software, hardware, or a combination of both. The device 1 includes a 3D data acquisition unit 11, an image preprocessing unit 15, a 2D data acquisition unit 12, a model training unit 16, a pose prediction unit 13, and a registration processing unit 14.

[0133] A three-dimensional data acquisition unit 11 is used to acquire three-dimensional imaging data of a target person;

[0134] An image pre-processing unit 15 is configured to obtain an initial two-dimensional scan image of the target person;

[0135] The initial two-dimensional scanned image is normalized and inverted to obtain a two-dimensional scanned image.

[0136] A two-dimensional data acquisition unit 12 is used to acquire a two-dimensional scan image of the target person and a machine feedback posture corresponding to the two-dimensional scan image;

[0137] The model training unit 16 is used to create an initial pose calculation model, obtain a sample two-dimensional scan image, and a sample machine feedback pose and a sample real pose corresponding to the sample two-dimensional scan image;

[0138] Inputting the sample two-dimensional scan image and the sample machine feedback pose into the initial pose calculation model to obtain training pose offset data output by the initial pose calculation model;

[0139] Based on the training pose offset data and the sample machine feedback pose, obtaining a training predicted pose corresponding to the sample two-dimensional scanned image;

[0140] Based on the training predicted pose and the actual pose of the sample, parameter adjustment processing is performed on the initial pose calculation model until model training is completed to obtain a pose calculation model.

[0141] Optionally, the model training unit 16 is specifically configured to generate a simulated two-dimensional scanned image based on the training predicted pose;

[0142] Based on the training predicted posture and the sample real posture, as well as the simulated two-dimensional scan image and the sample two-dimensional scan image, the initial posture calculation model is parameter adjusted until the model training is completed to obtain a posture calculation model.

[0143] Optionally, the model training unit 16 is specifically configured to obtain a first loss value based on the training predicted pose and the true pose of the sample;

[0144] obtaining a second loss value based on the simulated two-dimensional scan image and the sample two-dimensional scan image;

[0145] A summary loss value is obtained based on the first loss value and the second loss value, and parameter adjustment processing is performed on the initial pose calculation model based on the summary loss value until model training is completed to obtain a pose calculation model.

[0146] Optionally, the model training unit 16 is specifically configured to obtain a third loss value based on the sample machine feedback pose and the training predicted pose;

[0147] A summary loss value is obtained based on the first loss value, the second loss value and the third loss value, and parameter adjustment processing is performed on the initial pose calculation model based on the summary loss value until model training is completed to obtain a pose calculation model.

[0148] A posture prediction unit 13 is configured to use a posture calculation model to obtain a predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture;

[0149] Optionally, the posture prediction unit 13 is specifically configured to input the two-dimensional scanned image and the machine feedback posture into a posture calculation model to obtain posture offset data output by the posture calculation model;

[0150] Based on the posture offset data and the machine feedback posture, a predicted posture corresponding to the two-dimensional scanned image is obtained.

[0151] The registration processing unit 14 is configured to perform registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture.

[0152] Optionally, the registration processing unit 14 is specifically configured to perform rigid transformation processing on the three-dimensional imaging data based on the predicted posture, so as to register the three-dimensional imaging data with the two-dimensional scanned image.

[0153] In this embodiment, three-dimensional imaging data for the target person is obtained, an initial two-dimensional scanned image for the target person is obtained, and the initial two-dimensional scanned image is normalized and inverted to obtain a two-dimensional scanned image. The normalization and inversion processes can highlight the key features and dark details of the two-dimensional scanned image, thereby improving the generalization ability of the pose calculation model, reducing the risk of overfitting, and improving the computational efficiency and accuracy of the predicted pose. A two-dimensional scanned image for the target person and a machine feedback pose corresponding to the two-dimensional scanned image are obtained, an initial pose calculation model is created, a sample two-dimensional scanned image is obtained, and a sample machine feedback pose and a sample true pose corresponding to the sample two-dimensional scanned image are obtained. The sample two-dimensional scanned image and the sample machine feedback pose are input into the initial pose calculation model to obtain the training pose offset data output by the initial pose calculation model. Based on the training pose offset data and the sample machine feedback pose, the training predicted pose corresponding to the sample two-dimensional scanned image is obtained. Based on the training predicted pose and the sample true pose, the training predicted pose is obtained. The initial pose calculation model is parameter-adjusted until model training is completed to obtain a pose calculation model. The initial pose calculation model is trained using sample two-dimensional scanned images and sample machine feedback poses to obtain a pose calculation model. The pose calculation model is used to obtain a predicted pose of the two-dimensional scanned image, thereby improving the efficiency and accuracy of the predicted pose calculation. Moreover, by combining the first loss value, the second loss value, and the third loss value, the initial pose calculation model can learn and optimize the pose prediction ability from multiple perspectives, reducing the model's overfitting to the training data and improving its generalization ability. A constraint mechanism is applied during the model training process to ensure that the model output is within a certain range centered on the machine feedback pose, thereby avoiding falling into a local optimal solution and further improving the accuracy of the predicted pose. Using the pose calculation model, a predicted pose corresponding to the two-dimensional scanned image is obtained based on the two-dimensional scanned image and the machine feedback pose. Based on the predicted pose, a rigid transformation is performed on the three-dimensional imaging data to align the three-dimensional imaging data with the two-dimensional scanned image. The rigid transformation maintains the structural integrity of the three-dimensional imaging data, reduces error accumulation, simplifies the registration process, and improves the overall performance and reliability of image registration. By using the machine feedback posture as the initial posture and combining it with the posture calculation model to calculate the predicted posture, the result of the registration algorithm is approached to the global optimal solution, which can avoid falling into the local optimal solution and improve the registration efficiency and accuracy.

[0154] It should be noted that the image registration device provided in the above embodiment, when executing the image registration method, only uses the division of the above functional modules as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image registration device provided in the above embodiment and the image registration method embodiment are based on the same concept. The implementation process is detailed in the method embodiment and will not be repeated here.

[0155] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0156] The present application also provides a computer storage medium that can store multiple instructions, which are suitable for being loaded and executed by a processor as described above. Figures 1-10 The image registration method of the embodiment shown in the figure can be found in the specific implementation process. Figures 1-10 The detailed description of the illustrated embodiment will not be repeated here.

[0157] The present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor as described above. Figures 1-10 The image registration method of the embodiment shown in the figure can be found in the specific implementation process. Figures 1-10 The detailed description of the illustrated embodiment will not be repeated here.

[0158] Please refer to Figure 13 , which shows a block diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. The electronic device in the present application may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected via the bus 150.

[0159] The processor 110 may include one or more processing cores. The processor 110 utilizes various interfaces and circuits to connect various components within the electronic device. It executes instructions, programs, code sets, or instruction sets stored in the memory 120, as well as accesses data stored in the memory 120, to perform various terminal functions and process data. Optionally, the processor 110 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 110 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interfaces, and applications; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may also be implemented independently of the processor 110 via a separate communications chip.

[0160] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable storage medium (Non-Transitory Computer-Readable Storage Medium). The memory 120 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The operating system may be an Android system, including a system deeply developed based on the Android system, an iOS system developed by Apple, including a system deeply developed based on the iOS system, or other systems.

[0161] The memory 120 can be divided into an operating system space and a user space. The operating system runs in the operating system space, and native and third-party applications run in the user space. In order to ensure that different third-party applications can achieve better operating results, the operating system allocates corresponding system resources to different third-party applications. However, the requirements for system resources in different application scenarios in the same third-party application are also different. For example, in the local resource loading scenario, the third-party application has higher requirements for disk reading speed; in the animation rendering scenario, the third-party application has higher requirements for GPU performance. The operating system and the third-party application are independent of each other, and the operating system often cannot perceive the current application scenario of the third-party application in a timely manner, resulting in the operating system being unable to perform targeted system resource adaptation according to the specific application scenario of the third-party application.

[0162] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0163] The input device 130 is used to receive input commands or data and includes, but is not limited to, a keyboard, a mouse, a camera, a microphone, or a touch-sensitive device. The output device 140 is used to output commands or data and includes, but is not limited to, a display device and a speaker. In one example, the input device 130 and the output device 140 may be combined, and the input device 130 and the output device 140 may be a touch-sensitive display.

[0164] The touch display screen can be designed as a full screen, a curved screen or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the present embodiment.

[0165] In addition, those skilled in the art will understand that the structures of the electronic devices shown in the above figures do not limit the electronic devices. The electronic devices may include more or fewer components than shown, or may combine certain components or arrange the components differently. For example, the electronic devices may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WiFi) modules, power supplies, Bluetooth modules, and other components, which will not be described in detail here.

[0166] exist Figure 13 In the electronic device shown, the processor 110 may be configured to call an image registration application stored in the memory 120 and specifically perform the following operations:

[0167] Acquire three-dimensional imaging data of the target person;

[0168] Acquire a two-dimensional scanned image of the target person and a machine feedback posture corresponding to the two-dimensional scanned image;

[0169] Using a posture calculation model, based on the two-dimensional scanned image and the machine feedback posture, a predicted posture corresponding to the two-dimensional scanned image is obtained;

[0170] The three-dimensional imaging data and the two-dimensional scanned image are registered based on the predicted posture.

[0171] In one embodiment, before executing the posture calculation model to obtain the predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture, the processor 110 further performs the following operations:

[0172] Creating an initial pose calculation model, obtaining a sample two-dimensional scanned image, and a sample machine feedback pose and a sample true pose corresponding to the sample two-dimensional scanned image;

[0173] Inputting the sample two-dimensional scan image and the sample machine feedback pose into the initial pose calculation model to obtain training pose offset data output by the initial pose calculation model;

[0174] Based on the training pose offset data and the sample machine feedback pose, obtaining a training predicted pose corresponding to the sample two-dimensional scanned image;

[0175] Based on the training predicted pose and the actual pose of the sample, parameter adjustment processing is performed on the initial pose calculation model until model training is completed to obtain a pose calculation model.

[0176] In one embodiment, when the processor 110 performs parameter adjustment processing on the initial pose calculation model based on the training predicted pose and the sample true pose until the model training is completed and the pose calculation model is obtained, the processor 110 specifically performs the following operations:

[0177] generating a simulated two-dimensional scanned image based on the training predicted pose;

[0178] Based on the training predicted posture and the sample real posture, as well as the simulated two-dimensional scan image and the sample two-dimensional scan image, the initial posture calculation model is parameter adjusted until the model training is completed to obtain a posture calculation model.

[0179] In one embodiment, when the processor 110 performs parameter adjustment processing on the initial pose calculation model based on the training predicted pose and the sample real pose, as well as the simulated two-dimensional scanned image and the sample two-dimensional scanned image until the model training is completed and the pose calculation model is obtained, the processor 110 specifically performs the following operations:

[0180] Obtaining a first loss value based on the training predicted pose and the true pose of the sample;

[0181] obtaining a second loss value based on the simulated two-dimensional scan image and the sample two-dimensional scan image;

[0182] A summary loss value is obtained based on the first loss value and the second loss value, and parameter adjustment processing is performed on the initial pose calculation model based on the summary loss value until model training is completed to obtain a pose calculation model.

[0183] In one embodiment, when the processor 110 obtains a summary loss value based on the first loss value and the second loss value, and performs parameter adjustment processing on the initial pose calculation model based on the summary loss value until model training is completed to obtain the pose calculation model, the processor 110 specifically performs the following operations:

[0184] Obtaining a third loss value based on the sample machine feedback pose and the training predicted pose;

[0185] A summary loss value is obtained based on the first loss value, the second loss value and the third loss value, and parameter adjustment processing is performed on the initial pose calculation model based on the summary loss value until model training is completed to obtain a pose calculation model.

[0186] In one embodiment, before acquiring the two-dimensional scan image of the target person and the machine feedback posture corresponding to the two-dimensional scan image, the processor 110 further performs the following operations:

[0187] Acquire an initial two-dimensional scan image of the target person;

[0188] The initial two-dimensional scanned image is normalized and inverted to obtain a two-dimensional scanned image.

[0189] In one embodiment, when the processor 110 uses the posture calculation model to obtain the predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture, the processor 110 specifically performs the following operations:

[0190] Inputting the two-dimensional scanned image and the machine feedback posture into a posture calculation model to obtain posture offset data output by the posture calculation model;

[0191] Based on the posture offset data and the machine feedback posture, a predicted posture corresponding to the two-dimensional scanned image is obtained.

[0192] In one embodiment, when performing registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted pose, the processor 110 specifically performs the following operations:

[0193] A rigid transformation process is performed on the three-dimensional imaging data based on the predicted posture to align the three-dimensional imaging data with the two-dimensional scanned image.

[0194] In this embodiment, three-dimensional imaging data for the target person is obtained, an initial two-dimensional scanned image for the target person is obtained, and the initial two-dimensional scanned image is normalized and inverted to obtain a two-dimensional scanned image. The normalization and inversion processes can highlight the key features and dark details of the two-dimensional scanned image, thereby improving the generalization ability of the pose calculation model, reducing the risk of overfitting, and improving the computational efficiency and accuracy of the predicted pose. A two-dimensional scanned image for the target person and a machine feedback pose corresponding to the two-dimensional scanned image are obtained, an initial pose calculation model is created, a sample two-dimensional scanned image is obtained, and a sample machine feedback pose and a sample true pose corresponding to the sample two-dimensional scanned image are obtained. The sample two-dimensional scanned image and the sample machine feedback pose are input into the initial pose calculation model to obtain the training pose offset data output by the initial pose calculation model. Based on the training pose offset data and the sample machine feedback pose, the training predicted pose corresponding to the sample two-dimensional scanned image is obtained. Based on the training predicted pose and the sample true pose, the training predicted pose is obtained. The initial pose calculation model is parameter-adjusted until model training is completed to obtain a pose calculation model. The initial pose calculation model is trained using sample two-dimensional scanned images and sample machine feedback poses to obtain a pose calculation model. The pose calculation model is used to obtain a predicted pose of the two-dimensional scanned image, thereby improving the efficiency and accuracy of the predicted pose calculation. Moreover, by combining the first loss value, the second loss value, and the third loss value, the initial pose calculation model can learn and optimize the pose prediction ability from multiple perspectives, reducing the model's overfitting to the training data and improving its generalization ability. A constraint mechanism is applied during the model training process to ensure that the model output is within a certain range centered on the machine feedback pose, thereby avoiding falling into a local optimal solution and further improving the accuracy of the predicted pose. Using the pose calculation model, a predicted pose corresponding to the two-dimensional scanned image is obtained based on the two-dimensional scanned image and the machine feedback pose. Based on the predicted pose, a rigid transformation is performed on the three-dimensional imaging data to align the three-dimensional imaging data with the two-dimensional scanned image. The rigid transformation maintains the structural integrity of the three-dimensional imaging data, reduces error accumulation, simplifies the registration process, and improves the overall performance and reliability of image registration. By using the machine feedback posture as the initial posture and combining it with the posture calculation model to calculate the predicted posture, the result of the registration algorithm is approached to the global optimal solution, which can avoid falling into the local optimal solution and improve the registration efficiency and accuracy.

[0195] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0196] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

[0197] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, and display, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the three-dimensional imaging data and two-dimensional scanned images mentioned in this specification were all obtained with full authorization.

Claims

1. An image registration method, characterized in that: The method comprises: Acquire three-dimensional imaging data of the target person; Acquire a two-dimensional scanned image of the target person and a machine-feedback posture corresponding to the two-dimensional scanned image, wherein the machine-feedback posture is the posture corresponding to the acquisition of the two-dimensional scanned image as fed back by a two-dimensional image acquisition machine; Using a posture calculation model, based on the two-dimensional scanned image and the machine feedback posture, a predicted posture corresponding to the two-dimensional scanned image is obtained; Performing registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture; The adopting the posture calculation model to obtain the predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture includes: Inputting the two-dimensional scanned image and the machine feedback posture into a posture calculation model to obtain posture offset data output by the posture calculation model; Based on the posture offset data and the machine feedback posture, a predicted posture corresponding to the two-dimensional scanned image is obtained.

2. The method according to claim 1, characterized in that Before acquiring the predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture using the posture calculation model, the method further includes: Creating an initial pose calculation model, obtaining a sample two-dimensional scanned image, and a sample machine feedback pose and a sample true pose corresponding to the sample two-dimensional scanned image; Inputting the sample two-dimensional scan image and the sample machine feedback pose into the initial pose calculation model to obtain training pose offset data output by the initial pose calculation model; Based on the training pose offset data and the sample machine feedback pose, obtaining a training predicted pose corresponding to the sample two-dimensional scanned image; Based on the training predicted pose and the actual pose of the sample, parameter adjustment processing is performed on the initial pose calculation model until model training is completed to obtain a pose calculation model.

3. The method according to claim 2, characterized in that The step of adjusting parameters of the initial pose calculation model based on the training predicted pose and the sample true pose until model training is completed to obtain a pose calculation model includes: generating a simulated two-dimensional scanned image based on the training predicted pose; Based on the training predicted posture and the sample real posture, as well as the simulated two-dimensional scan image and the sample two-dimensional scan image, the initial posture calculation model is parameter adjusted until the model training is completed to obtain a posture calculation model.

4. The method according to claim 3, characterized in that The method further comprises: performing parameter adjustment processing on the initial pose calculation model based on the training predicted pose and the sample real pose, as well as the simulated two-dimensional scanned image and the sample two-dimensional scanned image until the model training is completed to obtain the pose calculation model, including: Obtaining a first loss value based on the training predicted pose and the true pose of the sample; obtaining a second loss value based on the simulated two-dimensional scan image and the sample two-dimensional scan image; A summary loss value is obtained based on the first loss value and the second loss value, and parameter adjustment processing is performed on the initial pose calculation model based on the summary loss value until model training is completed to obtain a pose calculation model.

5. The method according to claim 4, characterized in that The step of obtaining a summary loss value based on the first loss value and the second loss value, and performing parameter adjustment processing on the initial pose calculation model based on the summary loss value until model training is completed to obtain a pose calculation model includes: Obtaining a third loss value based on the sample machine feedback pose and the training predicted pose; A summary loss value is obtained based on the first loss value, the second loss value and the third loss value, and parameter adjustment processing is performed on the initial pose calculation model based on the summary loss value until model training is completed to obtain a pose calculation model.

6. The method according to claim 1, characterized in that Before obtaining the two-dimensional scanned image of the target person and the machine feedback posture corresponding to the two-dimensional scanned image, the method further includes: Acquire an initial two-dimensional scan image of the target person; The initial two-dimensional scanned image is normalized and inverted to obtain a two-dimensional scanned image.

7. The method according to claim 1, characterized in that The registering process of the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture includes: A rigid transformation process is performed on the three-dimensional imaging data based on the predicted posture to align the three-dimensional imaging data with the two-dimensional scanned image.

8. An image registration device, characterized in that: The device comprises: A three-dimensional data acquisition unit, used to acquire three-dimensional imaging data of a target person; a two-dimensional data acquisition unit, configured to acquire a two-dimensional scanned image of the target person and a machine-feedback posture corresponding to the two-dimensional scanned image, wherein the machine-feedback posture is the posture corresponding to the acquisition of the two-dimensional scanned image as fed back by a two-dimensional image acquisition machine; A posture prediction unit, configured to adopt a posture calculation model to obtain a predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture; a registration processing unit, configured to perform registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture; The posture prediction unit is specifically configured to input the two-dimensional scanned image and the machine feedback posture into a posture calculation model to obtain posture offset data output by the posture calculation model; Based on the posture offset data and the machine feedback posture, a predicted posture corresponding to the two-dimensional scanned image is obtained.

9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps according to any one of claims 1 to 7.

Images