Image registration method and device, storage medium and electronic equipment

Abstract

The invention provides an image registration method and device, a storage medium and electronic equipment, and the method is applied to the technical field of medical image processing, and comprises the steps: obtaining three-dimensional imaging data for a target person, obtaining a two-dimensional scanning image for the target person, and obtaining a machine feedback pose corresponding to the two-dimensional scanning image, and acquiring a predicted pose corresponding to the two-dimensional scanning image based on the two-dimensional scanning image and the machine feedback pose by adopting a pose calculation model, and performing registration processing on the three-dimensional imaging data and the two-dimensional scanning image based on the predicted pose. According to the method, the machine feedback pose is taken as the initial pose, and the predicted pose is calculated by combining the pose calculation model, so that the result of the registration algorithm approaches to the globally optimal solution and can be prevented from falling into the locally optimal solution, and the registration efficiency and the registration precision are improved.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and more specifically, to an image registration method, device, storage medium and electronic device in the technical field of medical image processing. Background Art

[0002] During the operation, accurately locating the surgical position is very important for the success and effect of the operation. Therefore, in order to accurately locate the surgical position and improve the surgical accuracy, the existing technology can combine two-dimensional X-ray images and three-dimensional cone-beam computerized tomography images to perform surgical navigation and locate the surgical position. However, only registered X-ray images and three-dimensional cone-beam computerized tomography images can achieve accurate positioning. If three-dimensional cone-beam computerized tomography images are collected from the human body during the operation, radiation leakage will result. If a gradient-based iterative optimization algorithm is used to achieve alignment, it is necessary to find the optimal solution through multiple iterations, which will consume a lot of computing power and a long calculation time. It is necessary to provide a method for quickly aligning two-dimensional X-ray images and three-dimensional cone-beam computerized tomography images. Summary of the invention

[0003] The embodiments of the present application provide an image registration method, device, storage medium and electronic device. The method can obtain the predicted posture by taking the machine feedback posture as the initial posture and combining it with the posture calculation model, so that the result of the registration algorithm approaches the global optimal solution, avoids falling into the local optimal solution, and improves the registration efficiency and accuracy.

[0004] In a first aspect, an embodiment of the present application provides an image registration method, the method comprising: Acquire three-dimensional imaging data of the target person; Acquire a two-dimensional scanned image of the target person and a machine feedback posture corresponding to the two-dimensional scanned image; Using a posture calculation model, based on the two-dimensional scanned image and the machine feedback posture, obtain a predicted posture corresponding to the two-dimensional scanned image; The three-dimensional imaging data and the two-dimensional scanned image are registered based on the predicted posture.

[0005] In a second aspect, an embodiment of the present application provides an image registration device, the device comprising: A three-dimensional data acquisition unit, used to acquire three-dimensional imaging data of a target person; A two-dimensional data acquisition unit, used to acquire a two-dimensional scan image of the target person and a machine feedback posture corresponding to the two-dimensional scan image; A posture prediction unit, configured to adopt a posture calculation model to obtain a predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture; A registration processing unit is used to perform registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture.

[0006] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0007] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0008] In one or more embodiments of the present application, three-dimensional imaging data for a target person is obtained, a two-dimensional scanned image for the target person, and a machine feedback posture corresponding to the two-dimensional scanned image are obtained, a posture calculation model is used, and a predicted posture corresponding to the two-dimensional scanned image is obtained based on the two-dimensional scanned image and the machine feedback posture, and the three-dimensional imaging data and the two-dimensional scanned image are registered based on the predicted posture. By using the machine feedback posture as the initial posture and combining the posture calculation model to calculate the predicted posture, the result of the registration algorithm approaches the global optimal solution, which can avoid falling into the local optimal solution, thereby improving the registration efficiency and registration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0010] Figure 1 It is a flowchart of an image registration method provided in an embodiment of the present application; Figure 2 It is a flowchart of a posture calculation model training method provided in an embodiment of the present application; Figure 3 It is a flowchart of a loss value calculation method provided in an embodiment of the present application; Figure 4 It is a flowchart of a loss value calculation method provided in an embodiment of the present application; Figure 5 This is an example schematic diagram of a model training and loss value calculation provided in an embodiment of the present application; Figure 6 It is a flowchart of an image preprocessing method provided in an embodiment of the present application; Figure 7 This is an example schematic diagram of a two-dimensional scan image preprocessing provided in an embodiment of the present application; Figure 8 It is a flowchart of a three-dimensional data preprocessing method provided in an embodiment of the present application; Fig. 9 It is a flowchart of an image preprocessing method provided in an embodiment of the present application; Fig.10 is a schematic diagram of an example of an image registration effect provided in an embodiment of the present application; Fig.11 is a structural schematic diagram of an image registration device provided in an embodiment of the present application; Fig.12 is a structural schematic diagram of an image registration device provided in an embodiment of the present application; Fig.13 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0012] In order to allow doctors to observe surgical instruments and surgical locations, three-dimensional imaging data and two-dimensional scanning images for surgical personnel, that is, patients, can be combined to guide surgery, thereby improving the safety and accuracy of surgery, reducing trauma to patients, and reducing the risk of complications. Among them, the three-dimensional imaging data can be three-dimensional cone-beam computed tomography (CBCT) data, and the two-dimensional scanning image can be a two-dimensional X-ray image. Among them, CBCT is a medical imaging technology that can use cone-beam X-rays to scan objects and reconstruct three-dimensional images through computers. For example, an X-ray generator can be used to perform circular digital projection around an object with a low amount of radiation. The X-ray source and detector rotate around the object, usually 360 degrees, to obtain two-dimensional projection images at multiple angles. These projection images are collected and reassembled by the computer to finally generate a three-dimensional image. X-ray images are an imaging technology that uses images formed after X-rays penetrate an object to display the internal structure of an object. When X-rays penetrate an object, tissues with different densities and atomic numbers, such as bones, muscles, fat, etc., absorb X-rays to different degrees. High-density tissues, such as bones, absorb more X-rays, while low-density tissues, such as air, absorb less X-rays. The X-rays that penetrate are received by the detector to form a grayscale image. The brightness of the image reflects the difference in the absorption of X-rays by the tissue, thereby showing the internal structure of the object. The three-dimensional imaging data and the two-dimensional scanned image need to be registered according to the pose in order to accurately locate the surgical position. The pose can be the position, angle, focal length and other information of the X-ray light source when taking the two-dimensional scanned image.

[0013] It is understandable that as the operation progresses and the surgical position changes, the doctor can collect two-dimensional scanning images for the surgeon multiple times to obtain real-time and accurate surgical positions. The postures of these two-dimensional scanning images may be different, but two-dimensional scanning images have certain limitations in the accuracy of displaying blood vessels and tissue structures. They can only provide anatomical images from a single perspective and cannot show the true structure of human tissues and organs in three-dimensional space. Although three-dimensional imaging data is more accurate, it involves high doses of X-rays. If it is used frequently during surgery, the collection of three-dimensional imaging data for two-dimensional scanning images of different postures and their registration will cause patients and medical staff to be exposed to higher radiation.

[0014] The embodiment of the present application provides an image registration device, which can collect three-dimensional imaging data of a target person before surgery, collect real-time two-dimensional scanned images of the target person during surgery, and place the three-dimensional imaging data and the two-dimensional scanned image in the same coordinate system, and move and transform the three-dimensional imaging data based on the posture of the two-dimensional scanned image, so as to achieve the registration of the three-dimensional imaging data and the two-dimensional scanned image. The image registration method provided in the embodiment of the present application can be implemented by relying on a computer program and can be run on an image registration device based on the von Neumann system. The computer program can be integrated in an application or run as an independent tool application.

[0015] The image registration method provided by the present application is described in detail below in conjunction with specific embodiments.

[0016] See also Figure 1 , which is a flow chart of an image registration method provided in an embodiment of the present application. Figure 1 As shown, the method in the embodiment of the present application may include the following steps S101-S104.

[0017] S101, acquiring three-dimensional imaging data of a target person.

[0018] Specifically, since the collection of three-dimensional imaging data involves high-dose X-rays, three-dimensional imaging data can be collected in advance for the target person before the operation, and the image registration device can directly obtain the pre-collected three-dimensional imaging data for the target person, where the target person can be a patient who needs to undergo surgery.

[0019] S102, obtaining a two-dimensional scan image of a target person and a machine feedback posture corresponding to the two-dimensional scan image.

[0020] Specifically, as the surgical process progresses, the surgical position and the position of the medical device change, in order to obtain real-time surgical conditions, doctors and others can use a two-dimensional image acquisition machine to collect a two-dimensional scan image for the target person, wherein the two-dimensional image acquisition machine can be used to collect two-dimensional scan images such as X-ray images, for example, the two-dimensional image acquisition machine can be an X-ray machine. After collecting the two-dimensional scan image, the doctor and other relevant staff can send a registration instruction to the image registration device to instruct the image registration device to register the collected two-dimensional scan image with the pre-collected three-dimensional imaging data. The image registration device can obtain a two-dimensional scan image for the target person, as well as the machine feedback posture fed back by the two-dimensional image acquisition machine. The machine feedback posture is the posture fed back by the two-dimensional image acquisition machine when the two-dimensional scan image is collected, and can include information such as the angle of the X-ray light source, the position of the X-ray light source, the focal length of the machine, the distance between the target person and the X-ray light source, and the distance between the target person and the detection plate.

[0021] Optionally, in order to improve the efficiency and accuracy of the registration process, the image registration device can establish a unified coordinate system to facilitate the alignment of the two-dimensional scanning images and three-dimensional imaging data obtained by different devices to the same reference frame. For example, the unified coordinate system can be a world coordinate system, which is a fixed three-dimensional coordinate system that can be used to define the absolute position of an object in space, wherein the origin of the unified coordinate system can be set by the image registration device or relevant medical staff, for example, it can be the center of the target person's body. The posture of all devices can be described relative to the unified coordinate system, so the machine feedback posture can be the posture information under the unified coordinate system, for example, the position of the X-ray light source can be the coordinate under the unified coordinate system, and the angle of the X-ray light source can be the angle under the unified coordinate system.

[0022] S103, using a posture calculation model, obtaining a predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture.

[0023] Specifically, the machine feedback posture actually has certain errors and is not the accurate posture of the two-dimensional scanned image. If the machine feedback posture is directly used for registration processing, the registration accuracy will be reduced, and it will be difficult to determine the accurate surgical position. However, if the corresponding posture is directly obtained based on the two-dimensional scanned image, it will not only require a large amount of calculation and calculation time, thereby delaying the surgical process, but may also fall into the local optimal solution and fail to obtain the global optimal solution and the accurate posture. The image registration device can train a posture calculation model, and the posture calculation model can use the machine feedback posture as the initial posture to predict the posture corresponding to the two-dimensional scanned image. Therefore, the image registration device can use the posture calculation model to obtain the predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture. The predicted posture is the posture output by the posture calculation model for the two-dimensional scanned image, which is the actual posture of the two-dimensional scanned image predicted by the posture calculation model.

[0024] Optionally, the predicted pose may be pose information in a unified coordinate system.

[0025] S104, performing registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture.

[0026] Specifically, the image registration device can also place the three-dimensional imaging data in a unified coordinate system, and then perform registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture, so as to facilitate doctors and the like to use the registered three-dimensional imaging data and the two-dimensional scanned image to perform surgery on the target person, wherein the registration processing can be to move the three-dimensional imaging data to a position matching the predicted position of the two-dimensional scanned image, and then use the forward projection algorithm (ForwardProjectionAlgorithm) to project the three-dimensional imaging data to obtain the registered image, and doctors and the like can use the registered image to guide surgery.

[0027] Optionally, the image registration device may place the three-dimensional imaging data in a unified coordinate system, and obtain the coordinates of each voxel in the three-dimensional imaging data in the unified coordinate system.

[0028] In an embodiment of the present application, three-dimensional imaging data for a target person is obtained, a two-dimensional scanned image for the target person, and a machine feedback posture corresponding to the two-dimensional scanned image are obtained, a posture calculation model is used, and a predicted posture corresponding to the two-dimensional scanned image is obtained based on the two-dimensional scanned image and the machine feedback posture, and the three-dimensional imaging data and the two-dimensional scanned image are registered based on the predicted posture. By using the machine feedback posture as the initial posture and combining the posture calculation model to calculate the predicted posture, the result of the registration algorithm approaches the global optimal solution, which can avoid falling into the local optimal solution, thereby improving the registration efficiency and registration accuracy.

[0029] See also Figure 2 , which is a flow chart of a posture calculation model training method provided in an embodiment of the present application, and before step S103, the following steps may also be included: S201, creating an initial posture calculation model, obtaining a sample two-dimensional scanned image, and a sample machine feedback posture and a sample real posture corresponding to the sample two-dimensional scanned image.

[0030] Specifically, the image registration device can create an initial pose calculation model, which can be a deep residual neural network (Residual Network 50, ResNet50). The initial pose calculation model can have the ability to predict the pose offset data between the pose corresponding to the two-dimensional scanned image and the machine feedback pose based on the machine feedback pose and the two-dimensional scanned image. The pose offset data can be the difference between the pose corresponding to the two-dimensional scanned image and the machine feedback pose. In order to further optimize the ability of the initial pose calculation model to obtain pose offset data, the image registration device can obtain sample two-dimensional scanned images, as well as sample machine feedback poses and sample true poses corresponding to the sample two-dimensional scanned images, to train the initial pose calculation model.

[0031] Optionally, the sample two-dimensional scanning image obtained by the image registration device can be a two-dimensional scanning image obtained by doctors and other relevant personnel based on a real human body. The sample machine feedback posture is the posture corresponding to the posture when the sample two-dimensional scanning image was collected, which is fed back by the two-dimensional image acquisition machine. The sample real posture can be the accurate posture matched by the image registration device according to the range of all activities of the two-dimensional image acquisition machine. Therefore, the calculation of the sample real posture requires a lot of computing power but has high accuracy.

[0032] Optionally, the sample two-dimensional scanned image may also be a digitally reconstructed radiograph (DRR), which generates a two-dimensional image similar to a traditional two-dimensional scanned image from digital three-dimensional imaging data by computer simulation of the attenuation and exposure process of X-rays. The image registration device may obtain the sample three-dimensional imaging data and the sample real pose, wherein the sample three-dimensional imaging data may be input by doctors and other relevant staff, and the sample real pose may be a plurality of poses obtained by the image registration device within the range of all activities of the two-dimensional image acquisition machine. The image registration device may input the sample three-dimensional imaging data and the sample real pose into a differentiable digitally reconstructed radiograph (DiffDRR), which is a DRR generator based on PyTorch. DiffDRR may generate a corresponding sample two-dimensional scanned image according to the sample real pose and the sample three-dimensional imaging data.

[0033] S202, inputting the sample two-dimensional scan image and the sample machine feedback posture into the initial posture calculation model to obtain the training posture offset data output by the initial posture calculation model.

[0034] Specifically, the initial pose calculation model can output pose offset data based on the two-dimensional scanned image and the machine feedback pose, and the pose offset data is the difference between the real pose of the two-dimensional scanned image and the machine feedback pose predicted by the initial pose calculation model. Therefore, the image registration device can input the sample two-dimensional scanned image and the sample machine feedback pose into the initial pose calculation model, so as to obtain the training pose offset data output by the initial pose calculation model, and the training pose offset data is the difference between the real pose of the sample two-dimensional scanned image and the sample machine feedback pose predicted by the initial pose calculation model.

[0035] S203, based on the training pose offset data and the sample machine feedback pose, obtain the training predicted pose corresponding to the sample two-dimensional scanned image.

[0036] Specifically, the image registration device can obtain the training predicted pose corresponding to the sample two-dimensional scanned image based on the training pose offset data and the sample machine feedback pose, and the training predicted pose is the true pose of the two-dimensional scanned image predicted by the initial pose calculation model. The image registration device can sum the training pose offset data and the sample machine feedback pose, and the obtained sum is the training predicted pose.

[0037] S204, based on the training predicted posture and the actual posture of the sample, the parameters of the initial posture calculation model are adjusted until the model training is completed to obtain the posture calculation model.

[0038] Specifically, the image registration device can perform model training on the initial pose calculation model based on the training predicted pose and the sample true pose, and adjust the parameters of the initial pose calculation model during the model training process to reduce the loss value between the training predicted pose and the sample true pose, so that the training predicted pose is closer to the sample true pose, thereby improving the accuracy of the initial pose calculation model, until the model training is completed and the pose calculation model is obtained.

[0039] Optionally, the image registration device performs model training on the initial pose calculation model until the initial pose calculation model meets the training termination condition, and then completes the model training to obtain the pose calculation model. The training termination condition is used to determine whether the initial pose calculation model has the ability to obtain accurate pose offset data. The training termination condition can be an initial setting of the image registration device, or can be set by the user or relevant staff.

[0040] Optionally, the training termination condition may be that the model training rounds for the initial pose calculation model have reached a preset round. For example, the preset rounds may be 10,000 times. Using training rounds as the training termination condition is intuitive and easy to understand, and does not require complex calculations or judgment conditions. It only needs to set the rounds at the beginning of training, making the management of the training process simpler.

[0041] Optionally, the training termination condition may include a loss value falling below a preset loss value and maintaining it for a preset time, wherein the preset loss value is used to determine whether the prediction accuracy of the initial pose calculation model has reached the standard, and the preset time is used to determine whether the stability of the initial pose calculation model has reached the standard, wherein the preset loss value and the preset time can be the initial settings of the image registration device, and can also be set by the user or relevant staff, for example, the preset loss value can be 5%, and the preset time can be 10 rounds of model training.

[0042] In an embodiment of the present application, an initial posture calculation model is created, a sample two-dimensional scanned image, and a sample machine feedback posture and a sample real posture corresponding to the sample two-dimensional scanned image are obtained, the sample two-dimensional scanned image and the sample machine feedback posture are input into the initial posture calculation model, the training posture offset data output by the initial posture calculation model is obtained, and the training predicted posture corresponding to the sample two-dimensional scanned image is obtained based on the training posture offset data and the sample machine feedback posture. Based on the training predicted posture and the sample real posture, the initial posture calculation model is parameter adjusted until the model training is completed to obtain the posture calculation model. The initial posture calculation model is trained by using the sample two-dimensional scanned image and the sample machine feedback posture to obtain the posture calculation model, and the predicted posture of the two-dimensional scanned image is obtained by using the posture calculation model, thereby improving the efficiency and accuracy of the predicted posture calculation.

[0043] In addition to obtaining the loss value between the training predicted pose and the sample's true pose, the image registration device can make the training predicted pose obtained during the model training process closer to the sample's true pose. In order to further improve the accuracy of the pose calculation model, the two-dimensional scanned image corresponding to the training predicted pose can also be simulated. If the two-dimensional scanned image corresponding to the training predicted pose is closer to the sample's true pose, it means that the training predicted pose is closer to the sample's true pose.

[0044] See also Figure 3 , a flowchart of a method for calculating a loss value is provided for an embodiment of the present application, and step S204 may include the following steps: S301, generating a simulated two-dimensional scanned image based on the training predicted posture.

[0045] Specifically, the image registration device can input the training predicted pose into DiffDRR, and DiffDRR can generate a simulated two-dimensional scanned image for the sample three-dimensional imaging data based on the training predicted pose. The difference between the simulated two-dimensional scanned image and the sample two-dimensional scanned image can also represent the difference between the training predicted pose and the sample real pose, which can be used to measure the prediction accuracy of the initial pose calculation model. Therefore, the image registration device can adjust the parameters of the initial pose calculation model based on the training predicted pose and the sample real pose, as well as the simulated two-dimensional scanned image and the sample two-dimensional scanned image until the model training is completed to obtain the pose calculation model.

[0046] S302, obtaining a first loss value based on the training predicted pose and the actual pose of the sample.

[0047] Specifically, the image registration device can obtain a first loss value based on the training predicted pose and the sample true pose. The first loss value can be the distance (Distance) between the training predicted pose and the sample true pose. For example, if the pre-trained predicted pose and the sample true pose both have multiple degrees of freedom, the pre-trained predicted pose and the sample true pose can be placed in a high-level space to calculate the distance between the point corresponding to the pre-trained predicted pose and the point corresponding to the sample true pose.

[0048] Optionally, the first loss value may be composed of a geodesic loss and a dual geodesic loss, and the formula is as follows:

[0049] in, is the first loss value, can be the geodesic loss, It can be a dual geodesic loss, which is usually used to measure the difference between two rotation matrices. Using geodesic loss as the loss function, that is, using the geodesic distance between the training predicted pose and the sample's true pose as the regression loss, we can give two rotation matrices , in three-dimensional space, rotations can be obtained through the special orthogonal group To express, the geodesic distance is a natural distance metric on this group, which reflects the shortest path between two rotations. The angular distance between two rotation matrices is calculated as follows:

[0050]

[0051] in, is The logarithmic mapping on The logarithmic mapping on can obtain the pose The geodesic loss on is given by:

[0052] The dual geodesic loss is an extension of the geodesic loss, which considers the difference between rotation and translation at the same time. In three-dimensional space, a complete posture can be described by a rotation matrix and a translation vector, and is described by a special Euclidean group. To indicate that it can be The geodesic distance with length units is set above. Based on the focal length f of the two-dimensional scanned image, the above angular distance can be converted into arc length. The formula is as follows:

[0053] The dual geodesic loss combines the geodesic distance of the rotation and the Euclidean distance between two translation vectors, so it is similar to the Euclidean distance of the translation. After combining, we can obtain The dual geodesic loss on is given by:

[0054] S303: Obtain a second loss value based on the simulated two-dimensional scanning image and the sample two-dimensional scanning image.

[0055] Specifically, the image registration device can obtain a second loss value based on the two-dimensional scanned image and the sample two-dimensional scanned image. The second loss value can be a multi-scale normalized cross-correlation (mNCC). By calculating the normalized cross-correlation at multiple scales, the features of the image at different scales can be captured, and the error that may be caused by single-scale analysis can be reduced, thereby improving the robustness and reliability of the algorithm. mNCC can compare the similarity between two images, and its value range is [-1, 1]. The closer the value of mNCC is to 1, the more similar the two images are.

[0056] S304, obtaining a summary loss value based on the first loss value and the second loss value, and performing parameter adjustment processing on the initial posture calculation model based on the summary loss value until the model training is completed to obtain the posture calculation model.

[0057] Specifically, the image registration device can combine the first loss value and the second loss value to obtain a summary loss value, and adjust the parameters of the initial pose calculation model during the model training process to reduce the summary loss value, thereby improving the accuracy of the initial pose calculation model, until the model training is completed and the pose calculation model is obtained.

[0058] Optionally, the image registration device can calculate the difference between 1 and the second loss value, and the product of the predicted ratio parameter and the first loss value, and then calculate the sum of the difference between 1 and the second loss value and the product of the predicted ratio parameter and the first loss value to obtain a summary loss value, wherein the predicted ratio parameter can be the initial setting of the image registration device, and can also be set by doctors and other relevant staff, for example, it can be 0.01.

[0059] In the embodiment of the present application, a simulated two-dimensional scanned image is generated based on the training predicted pose, a first loss value is obtained based on the training predicted pose and the sample real pose, a second loss value is obtained based on the simulated two-dimensional scanned image and the sample two-dimensional scanned image, a summary loss value is obtained based on the first loss value and the second loss value, and a parameter adjustment process is performed on the initial pose calculation model based on the summary loss value until the model training is completed to obtain a pose calculation model. By combining the first loss value and the second loss value, the initial pose calculation model can learn and optimize the pose prediction capability from multiple angles, reduce the overfitting of the model to the training data, and improve the generalization ability.

[0060] Although the machine feedback pose fed back by the two-dimensional image acquisition machine is not accurate, it is relatively close to the real pose corresponding to the two-dimensional scanned image. In order to avoid the pose calculation model from falling into the local optimal solution, a constraint mechanism can be introduced in the model training stage. During the model training process, the difference between the training predicted pose predicted by the initial pose calculation model and the real pose of the sample is calculated, thereby constraining the predicted pose to be within a certain range centered on the machine feedback pose.

[0061] See also Figure 4 , a flowchart of a method for calculating a loss value is provided for an embodiment of the present application, and step S304 may include the following steps: S401, obtaining a third loss value based on the sample machine feedback posture and the training predicted posture.

[0062] Specifically, the image registration device can obtain a third loss value based on the sample machine feedback pose and the training predicted pose. The third loss value can be the absolute value of the difference between the sample machine feedback pose and the training predicted pose, or it can be the absolute value of the training offset data.

[0063] S402, obtaining a summary loss value based on the first loss value, the second loss value and the third loss value, and performing parameter adjustment processing on the initial posture calculation model based on the summary loss value until the model training is completed to obtain the posture calculation model.

[0064] Specifically, the image registration device can combine the first loss value, the second loss value and the third loss value to obtain a summary loss value, and adjust the parameters of the initial pose calculation model during the model training process to reduce the summary loss value, thereby improving the accuracy of the initial pose calculation model until the model training is completed to obtain the pose calculation model.

[0065] Optionally, the summary loss value is calculated as follows:

[0066] in, is the summary loss value, is the second loss value, is the first loss value, is the third loss value.

[0067] Please also see Figure 5 , an example schematic diagram of model training and loss value calculation is provided for an embodiment of the present application. The image registration device can input the sample real pose into DiffDRR for simulation to obtain a sample two-dimensional scanned image, and input the sample two-dimensional scanned image and its corresponding sample machine feedback pose into the initial pose calculation model to obtain the training pose offset data output by the initial pose calculation model. The training pose offset data and the sample machine feedback pose can be combined to obtain the training predicted pose. The training predicted pose is input into DiffDRR for simulation to obtain a simulated two-dimensional scanned image. Then, a first loss value can be obtained based on the sample real pose and the training predicted pose, a second loss value can be obtained based on the sample two-dimensional scanned image and the simulated two-dimensional scanned image, and a third loss value can be obtained based on the training offset data. Then, a summary loss value can be obtained based on the first loss value, the second loss value and the third loss value.

[0068] In the embodiment of the present application, a third loss value is obtained based on the sample machine feedback posture and the training predicted posture, a summary loss value is obtained based on the first loss value, the second loss value and the third loss value, and the initial posture calculation model is parameter adjusted based on the summary loss value until the model training is completed to obtain the posture calculation model. By applying a constraint mechanism during the model training process, the model output is within a certain range centered on the machine feedback posture, thereby avoiding falling into a local optimal solution and further improving the accuracy of the predicted posture.

[0069] In order to further improve the accuracy of the pose calculation model, the two-dimensional scanned image can be preprocessed. During the preprocessing, the key features of the image can be enhanced, the noise can be removed, and the format and scale of the image can be standardized, so that the pose calculation model can efficiently learn the two-dimensional scanned image, thereby improving the reliability and stability of the predicted pose.

[0070] See also Figure 6 , which is a flowchart of an image preprocessing method according to an embodiment of the present application, and before step S102, the following steps may also be included: S501, obtaining an initial two-dimensional scan image of a target person.

[0071] Specifically, the image registration device can obtain an initial two-dimensional scan image of the target person, where the initial two-dimensional scan image is a two-dimensional scan image acquired by a two-dimensional image acquisition machine.

[0072] S502, performing normalization and inversion processing on the initial two-dimensional scanned image to obtain a two-dimensional scanned image.

[0073] Specifically, the image registration device can perform normalization processing on the initial two-dimensional scanned image, so as to normalize each pixel value in the initial two-dimensional scanned image to between 0 and 1, which can eliminate the problem of inconsistent pixel value ranges between different images due to factors such as lighting conditions and imaging equipment differences, and help highlight the key features in the image, so that the pose calculation model can more accurately identify and learn these features. Then, the grayscale value of the normalized initial two-dimensional scanned image can be inverted to obtain a two-dimensional scanned image. The inversion process can enhance the contrast of the image, making the dark features more obvious, thereby highlighting the dark details.

[0074] Optionally, the image registration device may also perform contrast limited adaptive histogram equalization on the initial two-dimensional scanned image after the inversion process, so as to obtain a two-dimensional scanned image. Figure 7 , an example schematic diagram of a two-dimensional scanned image preprocessing is provided for an embodiment of the present application. The image registration device can perform normalization processing, inversion processing, and contrast-limited adaptive histogram equalization and other preprocessing on the initial two-dimensional scanned image to obtain a two-dimensional scanned image.

[0075] In the embodiment of the present application, an initial two-dimensional scan image of the target person is obtained, and the initial two-dimensional scan image is normalized and inverted to obtain a two-dimensional scan image. The normalization and inversion processes can highlight the key features and dark details of the two-dimensional scan image, thereby improving the generalization ability of the posture calculation model, reducing the risk of overfitting, and improving the calculation efficiency and accuracy of the predicted posture.

[0076] In addition to preprocessing two-dimensional scanned images, the image registration device can also preprocess three-dimensional imaging data to improve registration accuracy and image clarity.

[0077] See also Figure 8 , a flow chart of a three-dimensional data preprocessing method is provided for an embodiment of the present application. In one or more embodiments of the present application, step S101 may include the following steps: S601, obtaining initial three-dimensional imaging data of a target person.

[0078] Specifically, the initial three-dimensional imaging data may be three-dimensional imaging data collected in advance for the target person before the operation.

[0079] S602, weight-increasing processing is performed on the skeleton parts in the initial three-dimensional imaging data, and weight-reducing processing is performed on the non-skeletal parts in the initial three-dimensional imaging data, so as to obtain three-dimensional imaging data for the target person.

[0080] Specifically, the Hounsfield Unit (HU) value is a quantitative unit used to describe the X-ray absorption rate of a substance, which can be used to distinguish tissues of different densities. The image registration device can obtain the HU value of each voxel in the initial three-dimensional imaging data, thereby determining the bone part and non-bone part in the initial three-dimensional imaging data, wherein the non-bone part can be air and soft tissue. Then, the weight of the bone part is increased and the weight of the non-bone part is decreased to obtain the three-dimensional imaging data, which reduces the interference of the non-bone part on the bone structure, enhances the contrast and clarity of the bone structure, and highlights the details of the bone structure. While improving the visualization effect of the three-dimensional imaging data, it also improves the efficiency and accuracy of image registration.

[0081] Optionally, the HU value can reflect the density and composition of the tissue. Different tissues have different HU value ranges, so the image registration device can determine the bone parts and non-bone parts in the initial three-dimensional imaging data based on the preset HU value. The parts with HU values ​​greater than the preset HU value can be determined as bone parts, and the parts with HU values ​​less than the preset HU value can be determined as non-bone parts. The preset HU value can be the initial setting of the image registration device, and can also be set by the user or relevant medical staff. Since the HU value of air is approximately -1000, indicating that air absorbs the least X-rays, the HU value of soft tissue can be between -100 and +100, such as muscle, fat, etc. The HU value of bones is higher, usually above +1000, such as cortical bone, so the preset HU value can be 1000.

[0082] In the embodiment of the present application, the initial three-dimensional imaging data for the target person is obtained, the weight of the bone parts in the initial three-dimensional imaging data is increased, and the weight of the non-bone parts in the initial three-dimensional imaging data is reduced to obtain the three-dimensional imaging data for the target person. By enhancing the contrast and clarity of the bone structure, the details of the bone structure are highlighted, and while improving the visualization effect of the three-dimensional imaging data, the efficiency and accuracy of image registration are also improved.

[0083] Since the posture calculation model can output posture offset data based on the two-dimensional scanned image and the machine feedback posture, rather than directly outputting the predicted posture, the image registration device needs to perform calculations based on the data output by the posture calculation model.

[0084] See also Fig. 9, a flowchart of an image preprocessing method is provided for an embodiment of the present application, and step S103 may include the following steps: S701, input the two-dimensional scanning image and the machine feedback posture into the posture calculation model to obtain the posture offset data output by the posture calculation model.

[0085] Specifically, the image registration device can input the two-dimensional scanned image and the machine feedback posture into the posture calculation model, and then obtain the posture offset data output by the posture calculation model. The posture offset data is the difference between the actual posture of the two-dimensional scanned image predicted by the posture calculation model and the machine feedback posture.

[0086] S702, based on the posture offset data and the machine feedback posture, obtain the predicted posture corresponding to the two-dimensional scanned image.

[0087] Specifically, the image registration device can calculate the predicted pose corresponding to the two-dimensional scanned image based on the pose offset data and the machine feedback pose. For example, the sum of the pose offset data and the machine feedback pose can be calculated to obtain the predicted pose. Fig.10 , an example schematic diagram of an image registration effect is provided for an embodiment of the present application. A checkerboard pattern is a visualization method for intuitively displaying the registration effect. The degree of alignment between the two images can be displayed alternately in the form of a checkerboard. The three-dimensional imaging data can be projected using a front projection algorithm, and a checkerboard pattern generated by the projection and the two-dimensional scanned image can be obtained, that is, the checkerboard pattern before image registration. It can be observed in the checkerboard pattern before image registration that there is an obvious deviation between the projection of the three-dimensional imaging data and the two-dimensional scanned image, while the projection of the three-dimensional imaging data in the checkerboard pattern after image registration has overlapped with the two-dimensional scanned image.

[0088] In the embodiment of the present application, the two-dimensional scanned image and the machine feedback posture are input into the posture calculation model, the posture offset data output by the posture calculation model is obtained, and the predicted posture corresponding to the two-dimensional scanned image is obtained based on the posture offset data and the machine feedback posture. By inputting the two-dimensional scanned image and the machine feedback posture into the posture calculation model, and calculating the predicted posture using the posture offset data output by the model, the accuracy and robustness of the predicted posture are improved.

[0089] In one or more embodiments of the present application, step S104 may include: The three-dimensional imaging data is rigidly transformed based on the predicted position and posture to align the three-dimensional imaging data with the two-dimensional scanned image.

[0090] Specifically, the image registration device can perform a rigid transformation on the three-dimensional imaging data in the unified coordinate system based on the predicted posture, so as to move the three-dimensional imaging data to a position that matches the predicted posture and is registered with the two-dimensional scanned image. Rigid transformation processing is a geometric transformation that can keep the shape and size of the three-dimensional imaging data unchanged and only change its position and direction, that is, rigid transformation processing will not change the distance between any two points in the three-dimensional imaging data, nor will it change the shape and volume of the three-dimensional imaging data. Rigid transformation processing consists of translation and rotation, and may also include reflection.

[0091] In the embodiment of the present application, the structural completeness of the three-dimensional imaging data is maintained through rigid transformation, error accumulation is reduced, the registration process is simplified, and the overall performance and reliability of image registration are improved.

[0092] The following will be combined with the attached Fig.11 -Attached Fig.12 , an image registration device provided in an embodiment of the present application is introduced in detail. It should be noted that the attached Fig.11 -Attached Fig.12 An image registration device in the present application is used to perform Figure 1-Figure 10 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 1-Figure 10 The embodiment shown.

[0093] See also Fig.11 , which shows a schematic diagram of the structure of an image registration device provided by an exemplary embodiment of the present application. The image registration device can be implemented as all or part of the device through software, hardware or a combination of both. The device 1 includes a three-dimensional data acquisition unit 11, a two-dimensional data acquisition unit 12, a posture prediction unit 13 and a registration processing unit 14.

[0094] A three-dimensional data acquisition unit 11, used to acquire three-dimensional imaging data of a target person; A two-dimensional data acquisition unit 12, used to acquire a two-dimensional scan image of the target person and a machine feedback posture corresponding to the two-dimensional scan image; A posture prediction unit 13, configured to use a posture calculation model to obtain a predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture; The registration processing unit 14 is used to perform registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture.

[0095] In this embodiment, three-dimensional imaging data for a target person is obtained, a two-dimensional scanned image for the target person, and a machine feedback posture corresponding to the two-dimensional scanned image are obtained, a posture calculation model is used, and a predicted posture corresponding to the two-dimensional scanned image is obtained based on the two-dimensional scanned image and the machine feedback posture, and the three-dimensional imaging data and the two-dimensional scanned image are registered based on the predicted posture. By taking the machine feedback posture as the initial posture and combining the posture calculation model to calculate the predicted posture, the result of the registration algorithm approaches the global optimal solution, which can avoid falling into the local optimal solution, thereby improving the registration efficiency and registration accuracy.

[0096] See also Fig.12 , which shows a schematic diagram of the structure of an image registration device provided by an exemplary embodiment of the present application. The image registration device can be implemented as all or part of the device through software, hardware or a combination of both. The device 1 includes a three-dimensional data acquisition unit 11, an image preprocessing unit 15, a two-dimensional data acquisition unit 12, a model training unit 16, a posture prediction unit 13 and a registration processing unit 14.

[0097] A three-dimensional data acquisition unit 11, used to acquire three-dimensional imaging data of a target person; An image preprocessing unit 15, used for acquiring an initial two-dimensional scan image of the target person; The initial two-dimensional scanned image is normalized and inverted to obtain a two-dimensional scanned image.

[0098] A two-dimensional data acquisition unit 12, used to acquire a two-dimensional scan image of the target person and a machine feedback posture corresponding to the two-dimensional scan image; A model training unit 16 is used to create an initial posture calculation model, obtain a sample two-dimensional scan image, and a sample machine feedback posture and a sample real posture corresponding to the sample two-dimensional scan image; Inputting the sample two-dimensional scanned image and the sample machine feedback posture into the initial posture calculation model to obtain the training posture offset data output by the initial posture calculation model; Based on the training pose offset data and the sample machine feedback pose, obtaining a training predicted pose corresponding to the sample two-dimensional scanned image; Based on the training predicted posture and the sample real posture, the initial posture calculation model is subjected to parameter adjustment processing until the model training is completed to obtain the posture calculation model.

[0099] Optionally, the model training unit 16 is specifically used to generate a simulated two-dimensional scanned image based on the training predicted posture; Based on the training predicted posture and the sample real posture, as well as the simulated two-dimensional scanned image and the sample two-dimensional scanned image, the initial posture calculation model is subjected to parameter adjustment processing until the model training is completed to obtain a posture calculation model.

[0100] Optionally, the model training unit 16 is specifically used to obtain a first loss value based on the training predicted posture and the actual posture of the sample; Obtaining a second loss value based on the simulated two-dimensional scanned image and the sample two-dimensional scanned image; A summary loss value is obtained based on the first loss value and the second loss value, and parameter adjustment processing is performed on the initial pose calculation model based on the summary loss value until model training is completed to obtain a pose calculation model.

[0101] Optionally, the model training unit 16 is specifically used to obtain a third loss value based on the sample machine feedback posture and the training predicted posture; A summary loss value is obtained based on the first loss value, the second loss value and the third loss value, and parameter adjustment processing is performed on the initial pose calculation model based on the summary loss value until the model training is completed to obtain a pose calculation model.

[0102] A posture prediction unit 13, configured to use a posture calculation model to obtain a predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture; Optionally, the posture prediction unit 13 is specifically used to input the two-dimensional scanned image and the machine feedback posture into a posture calculation model to obtain posture offset data output by the posture calculation model; Based on the posture offset data and the machine feedback posture, a predicted posture corresponding to the two-dimensional scanned image is obtained.

[0103] The registration processing unit 14 is used to perform registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture.

[0104] Optionally, the registration processing unit 14 is specifically used to perform rigid transformation processing on the three-dimensional imaging data based on the predicted posture, so as to align the three-dimensional imaging data with the two-dimensional scanned image.

[0105] In this embodiment, three-dimensional imaging data for the target person is obtained, an initial two-dimensional scanned image for the target person is obtained, and the initial two-dimensional scanned image is normalized and inverted to obtain a two-dimensional scanned image. The normalization and inversion processes can highlight the key features and dark details of the two-dimensional scanned image, thereby improving the generalization ability of the pose calculation model, reducing the risk of overfitting, and improving the calculation efficiency and accuracy of the predicted pose. The two-dimensional scanned image for the target person and the machine feedback pose corresponding to the two-dimensional scanned image are obtained, an initial pose calculation model is created, a sample two-dimensional scanned image, and a sample machine feedback pose and a sample true pose corresponding to the sample two-dimensional scanned image are obtained, the sample two-dimensional scanned image and the sample machine feedback pose are input into the initial pose calculation model, and the training pose offset data output by the initial pose calculation model are obtained. Based on the training pose offset data and the sample machine feedback pose, the training predicted pose corresponding to the sample two-dimensional scanned image is obtained. Based on the training predicted pose and the sample true pose, the training predicted pose is obtained. The method comprises the following steps: adjusting the parameters of the initial pose calculation model until the model training is completed to obtain the pose calculation model; training the initial pose calculation model by using the sample two-dimensional scanned image and the sample machine feedback pose to obtain the pose calculation model; using the pose calculation model to obtain the predicted pose of the two-dimensional scanned image, thereby improving the efficiency and accuracy of the predicted pose calculation; and combining the first loss value, the second loss value and the third loss value to enable the initial pose calculation model to learn and optimize the pose prediction ability from multiple angles, reduce the overfitting of the model to the training data, and improve the generalization ability; applying the constraint mechanism in the model training process to make the model output within a certain range centered on the machine feedback pose, thereby avoiding falling into the local optimal solution, and further improving the accuracy of the predicted pose. Using the pose calculation model, the predicted pose corresponding to the two-dimensional scanned image is obtained based on the two-dimensional scanned image and the machine feedback pose; rigid transformation processing is performed on the three-dimensional imaging data based on the predicted pose to align the three-dimensional imaging data with the two-dimensional scanned image; the structural completeness of the three-dimensional imaging data is maintained through rigid transformation, the error accumulation is reduced, the alignment process is simplified, and the overall performance and reliability of the image alignment are improved. By using the machine feedback posture as the initial posture and combining it with the posture calculation model to calculate the predicted posture, the result of the registration algorithm is approached to the global optimal solution, which can avoid falling into the local optimal solution and improve the registration efficiency and accuracy.

[0106] It should be noted that, when the image registration device provided in the above embodiment executes the image registration method, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the image registration device provided in the above embodiment and the image registration method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0107] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0108] The present application also provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figure 1-Figure 10 The image registration method of the embodiment shown in the figure can be specifically executed by referring to Figure 1-Figure 10 The specific description of the illustrated embodiment will not be repeated here.

[0109] The present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor as described above. Figure 1-Figure 10 The image registration method of the embodiment shown in the figure can be specifically executed by referring to Figure 1-Figure 10 The specific description of the illustrated embodiment will not be repeated here.

[0110] Please refer to Fig.13 , which shows a block diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. The electronic device in the present application may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected via the bus 150.

[0111] The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the terminal and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 110 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user pages, and applications; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0112] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable medium (Non-Transitory Computer-Readable Storage Medium). The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The operating system may be an Android system, including a system deeply developed based on the Android system, an IOS system developed by Apple, including a system deeply developed based on the IOS system, or other systems.

[0113] The memory 120 can be divided into an operating system space and a user space. The operating system runs in the operating system space, and native and third-party applications run in the user space. In order to ensure that different third-party applications can achieve good operating results, the operating system allocates corresponding system resources to different third-party applications. However, the requirements for system resources in different application scenarios in the same third-party application are also different. For example, in the local resource loading scenario, the third-party application has higher requirements for disk reading speed; in the animation rendering scenario, the third-party application has higher requirements for GPU performance. The operating system and third-party applications are independent of each other, and the operating system often cannot perceive the current application scenario of the third-party application in a timely manner, resulting in the operating system being unable to perform targeted system resource adaptation according to the specific application scenario of the third-party application.

[0114] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0115] The input device 130 is used to receive input commands or data, and includes but is not limited to a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is used to output commands or data, and includes but is not limited to a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 are touch screen displays.

[0116] The touch display screen can be designed as a full screen, a curved screen or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of the present application.

[0117] In addition, those skilled in the art can understand that the structure of the electronic device shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the drawings, or combine certain components, or arrange the components differently. For example, the electronic device also includes radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WiFi) modules, power supplies, Bluetooth modules and other components, which will not be described in detail here.

[0118] exist Fig.13 In the electronic device shown, the processor 110 may be used to call the image registration application stored in the memory 120, and specifically perform the following operations: Acquire three-dimensional imaging data of the target person; Acquire a two-dimensional scanned image of the target person and a machine feedback posture corresponding to the two-dimensional scanned image; Using a posture calculation model, based on the two-dimensional scanned image and the machine feedback posture, obtain a predicted posture corresponding to the two-dimensional scanned image; The three-dimensional imaging data and the two-dimensional scanned image are registered based on the predicted posture.

[0119] In one embodiment, before the processor 110 uses the posture calculation model to obtain the predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture, it further performs the following operations: Creating an initial posture calculation model, obtaining a sample two-dimensional scanned image, and a sample machine feedback posture and a sample real posture corresponding to the sample two-dimensional scanned image; Inputting the sample two-dimensional scanned image and the sample machine feedback posture into the initial posture calculation model to obtain the training posture offset data output by the initial posture calculation model; Based on the training pose offset data and the sample machine feedback pose, obtaining a training predicted pose corresponding to the sample two-dimensional scanned image; Based on the training predicted posture and the sample real posture, the initial posture calculation model is subjected to parameter adjustment processing until the model training is completed to obtain the posture calculation model.

[0120] In one embodiment, when the processor 110 performs parameter adjustment processing on the initial pose calculation model based on the training predicted pose and the sample real pose until the model training is completed and the pose calculation model is obtained, the following operations are specifically performed: Generate a simulated two-dimensional scanned image based on the training predicted pose; Based on the training predicted posture and the sample real posture, as well as the simulated two-dimensional scanned image and the sample two-dimensional scanned image, the initial posture calculation model is subjected to parameter adjustment processing until the model training is completed to obtain a posture calculation model.

[0121] In one embodiment, the processor 110 performs parameter adjustment processing on the initial pose calculation model based on the training predicted pose and the sample real pose, as well as the simulated two-dimensional scanned image and the sample two-dimensional scanned image until the model training is completed and the pose calculation model is obtained, and specifically performs the following operations: Obtaining a first loss value based on the training predicted pose and the sample true pose; Obtaining a second loss value based on the simulated two-dimensional scanned image and the sample two-dimensional scanned image; A summary loss value is obtained based on the first loss value and the second loss value, and parameter adjustment processing is performed on the initial pose calculation model based on the summary loss value until model training is completed to obtain a pose calculation model.

[0122] In one embodiment, when the processor 110 obtains a summary loss value based on the first loss value and the second loss value, and performs parameter adjustment processing on the initial pose calculation model based on the summary loss value until the model training is completed, and the pose calculation model is obtained, the following operations are specifically performed: Obtaining a third loss value based on the sample machine feedback posture and the training predicted posture; A summary loss value is obtained based on the first loss value, the second loss value and the third loss value, and parameter adjustment processing is performed on the initial pose calculation model based on the summary loss value until the model training is completed to obtain a pose calculation model.

[0123] In one embodiment, before acquiring the two-dimensional scanned image of the target person and the machine feedback posture corresponding to the two-dimensional scanned image, the processor 110 further performs the following operations: Acquire an initial two-dimensional scan image of the target person; The initial two-dimensional scanned image is normalized and inverted to obtain a two-dimensional scanned image.

[0124] In one embodiment, when the processor 110 uses the posture calculation model to obtain the predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture, the processor 110 specifically performs the following operations: Inputting the two-dimensional scanned image and the machine feedback posture into a posture calculation model to obtain posture offset data output by the posture calculation model; Based on the posture offset data and the machine feedback posture, a predicted posture corresponding to the two-dimensional scanned image is obtained.

[0125] In one embodiment, when the processor 110 performs the registration processing of the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture, the processor 110 specifically performs the following operations: The three-dimensional imaging data is subjected to rigid transformation processing based on the predicted posture so as to align the three-dimensional imaging data with the two-dimensional scanning image.

[0126] In this embodiment, three-dimensional imaging data for the target person is obtained, an initial two-dimensional scanned image for the target person is obtained, and the initial two-dimensional scanned image is normalized and inverted to obtain a two-dimensional scanned image. The normalization and inversion processes can highlight the key features and dark details of the two-dimensional scanned image, thereby improving the generalization ability of the pose calculation model, reducing the risk of overfitting, and improving the calculation efficiency and accuracy of the predicted pose. The two-dimensional scanned image for the target person and the machine feedback pose corresponding to the two-dimensional scanned image are obtained, an initial pose calculation model is created, a sample two-dimensional scanned image, and a sample machine feedback pose and a sample true pose corresponding to the sample two-dimensional scanned image are obtained, the sample two-dimensional scanned image and the sample machine feedback pose are input into the initial pose calculation model, and the training pose offset data output by the initial pose calculation model are obtained. Based on the training pose offset data and the sample machine feedback pose, the training predicted pose corresponding to the sample two-dimensional scanned image is obtained. Based on the training predicted pose and the sample true pose, the training predicted pose is obtained. The method comprises the following steps: adjusting the parameters of the initial pose calculation model until the model training is completed to obtain the pose calculation model; training the initial pose calculation model by using the sample two-dimensional scanned image and the sample machine feedback pose to obtain the pose calculation model; using the pose calculation model to obtain the predicted pose of the two-dimensional scanned image, thereby improving the efficiency and accuracy of the predicted pose calculation; and combining the first loss value, the second loss value and the third loss value to enable the initial pose calculation model to learn and optimize the pose prediction ability from multiple angles, reduce the overfitting of the model to the training data, and improve the generalization ability; applying the constraint mechanism in the model training process to make the model output within a certain range centered on the machine feedback pose, thereby avoiding falling into the local optimal solution, and further improving the accuracy of the predicted pose. Using the pose calculation model, the predicted pose corresponding to the two-dimensional scanned image is obtained based on the two-dimensional scanned image and the machine feedback pose; rigid transformation processing is performed on the three-dimensional imaging data based on the predicted pose to align the three-dimensional imaging data with the two-dimensional scanned image; the structural completeness of the three-dimensional imaging data is maintained through rigid transformation, the error accumulation is reduced, the alignment process is simplified, and the overall performance and reliability of the image alignment are improved. By using the machine feedback posture as the initial posture and combining it with the posture calculation model to calculate the predicted posture, the result of the registration algorithm is approached to the global optimal solution, which can avoid falling into the local optimal solution and improve the registration efficiency and accuracy.

[0127] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0128] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

[0129] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the three-dimensional imaging data, two-dimensional scanning images, etc. involved in this specification are all obtained with full authorization.

Claims

1. An image registration method, characterized in that: The method comprises: Acquire three-dimensional imaging data of the target person; Acquire a two-dimensional scanned image of the target person and a machine feedback posture corresponding to the two-dimensional scanned image; Using a posture calculation model, based on the two-dimensional scanned image and the machine feedback posture, obtain a predicted posture corresponding to the two-dimensional scanned image; The three-dimensional imaging data and the two-dimensional scanned image are registered based on the predicted posture.

2. The method according to claim 1, characterized in that Before the posture calculation model is used to obtain the predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture, the method further includes: Creating an initial posture calculation model, obtaining a sample two-dimensional scanned image, and a sample machine feedback posture and a sample real posture corresponding to the sample two-dimensional scanned image; Inputting the sample two-dimensional scanned image and the sample machine feedback posture into the initial posture calculation model to obtain the training posture offset data output by the initial posture calculation model; Based on the training pose offset data and the sample machine feedback pose, obtaining a training predicted pose corresponding to the sample two-dimensional scanned image; Based on the training predicted posture and the sample real posture, the initial posture calculation model is subjected to parameter adjustment processing until the model training is completed to obtain the posture calculation model.

3. The method according to claim 2, characterized in that The step of adjusting parameters of the initial posture calculation model based on the training predicted posture and the sample real posture until the model training is completed to obtain the posture calculation model includes: Generate a simulated two-dimensional scanned image based on the training predicted pose; Based on the training predicted posture and the sample real posture, as well as the simulated two-dimensional scanned image and the sample two-dimensional scanned image, the initial posture calculation model is subjected to parameter adjustment processing until the model training is completed to obtain a posture calculation model.

4. The method according to claim 3, characterized in that The method of performing parameter adjustment processing on the initial posture calculation model based on the training predicted posture and the sample real posture, as well as the simulated two-dimensional scanned image and the sample two-dimensional scanned image until the model training is completed to obtain the posture calculation model includes: Obtaining a first loss value based on the training predicted pose and the sample true pose; Obtaining a second loss value based on the simulated two-dimensional scanned image and the sample two-dimensional scanned image; A summary loss value is obtained based on the first loss value and the second loss value, and parameter adjustment processing is performed on the initial pose calculation model based on the summary loss value until model training is completed to obtain a pose calculation model.

5. The method according to claim 4, characterized in that The step of obtaining a summary loss value based on the first loss value and the second loss value, and performing parameter adjustment processing on the initial posture calculation model based on the summary loss value until model training is completed to obtain a posture calculation model includes: Obtaining a third loss value based on the sample machine feedback posture and the training predicted posture; A summary loss value is obtained based on the first loss value, the second loss value and the third loss value, and parameter adjustment processing is performed on the initial pose calculation model based on the summary loss value until model training is completed to obtain a pose calculation model.

6. The method according to claim 1, characterized in that Before obtaining the two-dimensional scanned image of the target person and the machine feedback posture corresponding to the two-dimensional scanned image, the method further includes: Acquire an initial two-dimensional scan image of the target person; The initial two-dimensional scanned image is normalized and inverted to obtain a two-dimensional scanned image.

7. The method according to claim 2, characterized in that: The adopting the posture calculation model to obtain the predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture includes: Inputting the two-dimensional scanned image and the machine feedback posture into a posture calculation model to obtain posture offset data output by the posture calculation model; Based on the posture offset data and the machine feedback posture, a predicted posture corresponding to the two-dimensional scanned image is obtained.

8. The method according to claim 1, characterized in that The registering process of the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture includes: The three-dimensional imaging data is subjected to rigid transformation processing based on the predicted posture so as to align the three-dimensional imaging data with the two-dimensional scanning image.

9. An image registration device, characterized in that: The device comprises: A three-dimensional data acquisition unit, used to acquire three-dimensional imaging data of a target person; A two-dimensional data acquisition unit, used to acquire a two-dimensional scan image of the target person and a machine feedback posture corresponding to the two-dimensional scan image; A posture prediction unit, configured to adopt a posture calculation model to obtain a predicted posture corresponding to the two-dimensional scanned image based on the two-dimensional scanned image and the machine feedback posture; A registration processing unit is used to perform registration processing on the three-dimensional imaging data and the two-dimensional scanned image based on the predicted posture.

10. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps as claimed in any one of claims 1 to 8.

11. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 8.

Images