Image matching method and device and pose acquisition model training method and device

Abstract

The application relates to an image matching method and device and a pose acquisition model training method and device. The method comprises the following steps: obtaining an original two-dimensional feature image to be matched and a projected two-dimensional feature image; the projected two-dimensional feature image is obtained through dimension reduction processing according to an original three-dimensional feature image to be matched; a first global pose is determined according to the projected two-dimensional feature image, and a second global pose is determined according to the original two-dimensional feature image and the projected two-dimensional feature image; a target global pose for the original three-dimensional feature image and the original two-dimensional feature image is determined based on the first global pose and the second global pose; and the target global pose is used for matching the original three-dimensional feature image and the original two-dimensional feature image. The method can improve the image matching precision.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image matching method, apparatus, and pose acquisition model training method and apparatus. Background Technology

[0002] In clinical diagnosis, it is often necessary to integrate information from multiple images so that doctors can understand the overall condition of diseased tissues or organs, make more accurate diagnoses, or formulate more appropriate treatment plans. The primary challenge in integrating multiple image information is ensuring strict image alignment, i.e., image registration. The effectiveness of image registration directly affects the quality of image fusion; therefore, only accurate registration can provide doctors with accurate diagnostic information.

[0003] Current methods for matching 3D and 2D feature images primarily rely on the relative motion relationship between 3D and 2D data. This involves directly fitting the relative motion between the 2D image reprojected from the 3D model and the actual 2D image, iterating through the fitting process to obtain the pose transformation for image matching. However, current matching methods for 3D and 2D feature images, which utilize both 3D and 2D data for matching or directly obtain the absolute pose from the 3D data, suffer from low accuracy due to the use of data from different modalities or the direct acquisition of absolute pose. Summary of the Invention

[0004] Therefore, it is necessary to provide an image matching method and apparatus, and a pose acquisition model training method and apparatus that can improve the accuracy of image matching, in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides an image matching method, which includes:

[0006] Obtain the original two-dimensional feature image and the projected two-dimensional feature image to be matched; the projected two-dimensional feature image is obtained by dimensionality reduction processing based on the original three-dimensional feature image to be matched.

[0007] The first global pose is determined based on the projected two-dimensional feature image, and the second global pose is determined based on the original two-dimensional feature image and the projected two-dimensional feature image.

[0008] Based on the first global pose and the second global pose, the target global pose is determined for the original 3D feature image and the original 2D feature image; the target global pose is used to match the original 3D feature image and the original 2D feature image.

[0009] In one embodiment, determining the second global pose based on the original two-dimensional feature image and the projected two-dimensional feature image includes:

[0010] The incremental pose is obtained based on the original two-dimensional feature image and the projected two-dimensional feature image;

[0011] Obtain the initial pose of the original 3D feature image, and obtain the second global pose based on the initial pose and the incremental pose.

[0012] In one embodiment, determining the first global pose based on the projected two-dimensional feature image includes:

[0013] The projected two-dimensional feature image is input into the pre-trained target pose acquisition model so that the target pose acquisition model performs first pose acquisition processing on the projected two-dimensional feature image to obtain the first global pose.

[0014] Based on the original two-dimensional feature image and the projected two-dimensional feature image, the incremental pose is obtained, including:

[0015] The original two-dimensional feature image is input into the target pose acquisition model so that the target pose acquisition model performs a second pose acquisition process on the original two-dimensional feature image and the projected two-dimensional feature image to obtain the incremental pose.

[0016] Secondly, this application provides a method for training a pose acquisition model, the method comprising:

[0017] Obtain the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample; the projected two-dimensional feature image of the sample is obtained by dimensionality reduction processing based on the three-dimensional feature image of the sample.

[0018] The original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample are input into the pose acquisition model to be trained, so that the pose acquisition model to be trained performs the first pose acquisition processing on the projected two-dimensional feature image of the sample to obtain the predicted first global pose; and performs the second pose acquisition processing on the original two-dimensional feature image and the projected two-dimensional feature image to obtain the predicted second global pose.

[0019] Based on the predicted first global pose and the predicted second global pose, determine the loss value of the model for the pose to be trained.

[0020] Based on the loss value of the pose acquisition model to be trained, the pose acquisition model to be trained is trained to obtain the target pose acquisition model.

[0021] In one embodiment, determining the loss value for the pose acquisition model to be trained based on the predicted first global pose and the predicted second global pose includes:

[0022] Obtain the first global pose of the sample for the 3D feature image of the sample, and determine the first loss value based on the predicted first global pose and the first global pose of the sample.

[0023] Obtain the second global pose of the sample based on the 3D feature image and the original 2D feature image of the sample, and determine the second loss value based on the predicted second global pose and the sample second global pose;

[0024] Based on the first loss value and the second loss value, determine the loss value for the pose acquisition model to be trained.

[0025] In one embodiment, determining the loss value for the pose acquisition model to be trained based on the predicted first global pose and the predicted second global pose includes:

[0026] Based on the predicted first global pose and the predicted second global pose, the predicted target global pose is obtained;

[0027] Based on the first global pose and the second global pose of the sample, the global pose of the target sample is obtained;

[0028] Based on the predicted global pose of the target and the global pose of the sample target, determine the loss value for the model to be trained based on the pose.

[0029] In one embodiment, a second pose acquisition process is performed on the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample to obtain a predicted second global pose, including:

[0030] The original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample are processed to obtain the predicted incremental pose.

[0031] The initial pose of the original 3D feature image of the sample is obtained, and the predicted second global pose is obtained based on the initial pose and the predicted incremental pose.

[0032] Thirdly, this application also provides an image matching apparatus. The apparatus includes:

[0033] The feature image acquisition module is used to acquire the original two-dimensional feature image to be matched and the projected two-dimensional feature image; the projected two-dimensional feature image is obtained by dimensionality reduction processing based on the original three-dimensional feature image to be matched.

[0034] The pose acquisition module is used to determine the first global pose based on the projected two-dimensional feature image, and to determine the second global pose based on the original two-dimensional feature image and the projected two-dimensional feature image.

[0035] The target pose acquisition module is used to determine the target global pose for the original 3D feature image and the original 2D feature image based on the first global pose and the second global pose; the target global pose is used to match the original 3D feature image and the original 2D feature image.

[0036] Fourthly, this application also provides a pose acquisition model training device, the device comprising:

[0037] The sample image acquisition module is used to acquire the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample; the projected two-dimensional feature image of the sample is obtained by dimensionality reduction processing based on the three-dimensional feature image of the sample.

[0038] The pose acquisition module is used to input the original two-dimensional feature image of the sample and the two-dimensional feature image of the sample projection into the pose acquisition model to be trained, so that the pose acquisition model to be trained performs a first pose acquisition process on the two-dimensional feature image of the sample projection to obtain a predicted first global pose; and performs a second pose acquisition process on the original two-dimensional feature image of the sample and the two-dimensional feature image of the sample projection to obtain a predicted second global pose.

[0039] The loss value determination module is used to determine the loss value of the model for the pose to be trained based on the predicted first global pose and the predicted second global pose.

[0040] The model training module is used to train the pose acquisition model to be trained based on the loss value of the pose acquisition model to be trained, so as to obtain the target pose acquisition model.

[0041] Fifthly, this application also provides a computer device. This computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0042] The aforementioned image matching method, apparatus, and pose acquisition model training method and apparatus acquire an original two-dimensional feature image and a projected two-dimensional feature image to be matched. The projected two-dimensional feature image is obtained by dimensionality reduction processing based on the original three-dimensional feature image to be matched. A first global pose is determined based on the projected two-dimensional feature image, and a second global pose is determined based on the original two-dimensional feature image and the projected two-dimensional feature image. Based on the first global pose and the second global pose, a target global pose is determined for the original three-dimensional feature image and the original two-dimensional feature image. The target global pose is used to match the original three-dimensional feature image and the original two-dimensional feature image. This application obtains a projected two-dimensional feature image by projecting the original three-dimensional feature image, obtains the first global pose through the projected two-dimensional feature image, obtains the second global pose through the original two-dimensional feature image and the projected two-dimensional feature image, and obtains the target global pose through the first global pose and the second global pose. Thus, by obtaining the first global pose and the second global pose separately through a multi-task method, the accuracy of obtaining the global pose can be improved, thereby improving the image matching accuracy. Attached Figure Description

[0043] Figure 1This is a diagram illustrating the application environment of an image matching method in one embodiment;

[0044] Figure 2 This is a flowchart illustrating an image matching method in one embodiment;

[0045] Figure 3 This is a flowchart illustrating a pose acquisition model training method in one embodiment;

[0046] Figure 4 This is a flowchart illustrating the steps for determining the loss value of a pose acquisition model to be trained in one embodiment.

[0047] Figure 5 This is a flowchart illustrating the pose acquisition model training method in another embodiment;

[0048] Figure 6 This is a structural diagram of a method for matching three-dimensional medical images and two-dimensional medical images in a specific embodiment;

[0049] Figure 7 This is a structural block diagram of an image matching device in one embodiment;

[0050] Figure 8 This is a structural block diagram of a pose acquisition model training device in one embodiment;

[0051] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] The image matching method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on other network servers. The data storage system can store raw 3D feature images and raw 2D feature images. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0054] In one embodiment, such as Figure 2 As shown, an image matching method is provided, which is applied to... Figure 1 Taking the server in the example, the following steps are included:

[0055] S202, obtain the original two-dimensional feature image to be matched and the projected two-dimensional feature image; the projected two-dimensional feature image is obtained by dimensionality reduction processing based on the original three-dimensional feature image to be matched.

[0056] The original 3D feature image can be a 3D image with a linear topological structure; the original 2D feature image can be a 2D image with a linear topological structure. A 3D image with a linear topological structure refers to an image that displays linear topological relationships. In other words, a 3D feature image is a 3D image that displays linear topological relationships, such as images of human blood vessels or human tissues. Similarly, an original 2D feature image is an image with the same feature structure as the original 3D feature image, the difference being that they have different dimensions. For example, a 3D feature image can be a preoperative medical image, and a 2D feature image can be an intraoperative medical image. The purpose of image matching is to register preoperative 3D medical images and intraoperative 2D medical images so that preoperative data has a corresponding relationship when used intraoperatively, facilitating surgery. Alternatively, in clinical diagnosis, it is often necessary to integrate information from multiple medical images of different dimensions so that doctors can understand the overall condition of diseased tissues or organs. In this case, medical image information integration requires the registration of medical images of different dimensions.

[0057] Here, the projected two-dimensional feature image refers to the two-dimensional feature image obtained after dimensionality reduction processing of the original three-dimensional feature image. Dimensionality reduction processing can be rigid body transformation processing, which can be spatial volume transformation. Rigid body transformation can be performed using preset pose parameters to obtain the projected two-dimensional feature image corresponding to the original three-dimensional feature image.

[0058] For example, at least one image acquisition device can be used to acquire the original three-dimensional feature image to be matched and the original two-dimensional feature image to be matched with the original three-dimensional feature image. The image acquisition device can be any device capable of acquiring both three-dimensional and two-dimensional feature images. For example, it can be a medical imaging device to acquire preoperative three-dimensional medical images and intraoperative two-dimensional medical images. For example, a three-dimensional medical image can be acquired using a three-dimensional medical imaging device, and a two-dimensional medical image can be acquired using a two-dimensional medical imaging device. By performing matching processing on the three-dimensional and two-dimensional medical images, the coordinate systems of the three-dimensional and two-dimensional medical imaging devices can be matched. Furthermore, all three-dimensional images in the three-dimensional medical imaging device and all two-dimensional images in the two-dimensional medical imaging device can be matched.

[0059] For example, the original 3D feature image is dimensionality reduced to obtain a corresponding projected 2D feature image. By performing a rigid body transformation on the original 3D feature image, a corresponding projected 2D feature image can be obtained. Furthermore, the projected 2D feature image and the original 2D feature image can be used to obtain relative pose parameters for matching. This avoids the problem of low matching accuracy caused by directly using 3D and 2D data from different modalities for matching. By obtaining the projected 2D feature image corresponding to the original 3D feature image, this application further improves the matching accuracy between the original 3D and 2D feature images.

[0060] S204, determine the first global pose based on the projected two-dimensional feature image, and determine the second global pose based on the original two-dimensional feature image and the projected two-dimensional feature image.

[0061] The first global pose refers to the global pose directly obtained by projecting a 2D feature image, which can be the absolute pose of the original 3D feature image. The second global pose refers to the global pose obtained after processing the image data of the original 2D feature image and the image data of the projected 2D feature image. The global pose refers to the pose parameters that can be used to match 3D and 2D feature images. For example, the 3D feature image can be processed to obtain a 2D feature image using the global pose, or the 3D feature image can be rotated or translated to obtain a 2D feature image. The 3D and 2D feature images are then matched using the global pose.

[0062] For example, the first global pose of the original three-dimensional feature image can be directly obtained using the image data of the projected two-dimensional feature image. Furthermore, a second global pose can be obtained by processing the image data of both the original two-dimensional feature image and the projected two-dimensional feature image. Thus, the first and second global poses can be obtained through a multi-task approach. Further, the two global poses can be fused, thereby improving the accuracy of global pose acquisition and consequently enhancing the matching accuracy between the original three-dimensional and original two-dimensional feature images.

[0063] For example, by projecting a two-dimensional feature image, the absolute pose of the original three-dimensional feature image can be obtained, and the relative pose of the original three-dimensional feature image and the projected two-dimensional feature image can be obtained through the original two-dimensional feature image and the projected two-dimensional feature image. This can solve the problem of scale ambiguity in image matching and the problem of low matching accuracy of different modal data.

[0064] S206, Based on the first global pose and the second global pose, obtain the target global pose for the original three-dimensional feature image and the original two-dimensional feature image; the target global pose is used to match the original three-dimensional feature image and the original two-dimensional feature image.

[0065] The target global pose refers to the global pose that can be used to match the original three-dimensional feature image with the original two-dimensional feature image. For example, the target global pose can be used to rotate or translate the original three-dimensional feature image to obtain the original three-dimensional feature image.

[0066] For example, the first global pose and the second global pose are fused to obtain a target global pose for matching the original 3D feature image and the original 2D feature image. Furthermore, this target global pose can be used to match other 3D feature images associated with the original 3D feature image and other original 2D feature images associated with the original 2D feature image. For example, the target global pose can be used to match preoperative 3D medical images and intraoperative 2D medical images. Thus, the target global pose obtained through multi-task processing, when applied to the original 3D feature image and the original 2D feature image, can improve the accuracy of image matching.

[0067] In this embodiment, the acquisition device acquires an original 3D feature image, an original 2D feature image, and a projected 2D feature image corresponding to the original 3D feature image. The projected 2D feature image is obtained by dimensionality reduction processing based on the original 3D feature image. A first global pose is determined based on the projected 2D feature image, and a second global pose is determined based on the original 2D feature image and the projected 2D feature image. A target global pose is determined based on the first global pose and the second global pose for the original 3D feature image and the original 2D feature image. The target global pose is used to match the original 3D feature image and the original 2D feature image. This application obtains a projected 2D feature image by projecting the original 3D feature image, obtains the first global pose through the projected 2D feature image, obtains the second global pose through the original 2D feature image and the projected 2D feature image, and obtains the target global pose through the first and second global poses. Thus, by acquiring the first and second global poses separately through a multi-task method, the accuracy of acquiring the global pose can be improved, thereby improving the image matching accuracy. Simultaneously, the multi-task processing method can improve the image matching scale accuracy and the matching accuracy of different modal data.

[0068] In one embodiment, determining the second global pose based on the original two-dimensional feature image and the projected two-dimensional feature image includes:

[0069] The incremental pose is obtained based on the original two-dimensional feature image and the projected two-dimensional feature image;

[0070] Obtain the initial pose of the original 3D feature image, and obtain the second global pose based on the initial pose and the incremental pose.

[0071] Incremental pose refers to the relative pose between the original 2D feature image and the projected 2D feature image. The initial pose can be obtained directly from the pose parameters of the original 2D feature image using a pre-defined image acquisition device; that is, the pose corresponding to the original 2D feature image. For example, the original 2D feature image can be acquired using a pre-defined image acquisition device, and simultaneously, the initial pose of the original 3D feature image can be obtained. This initial pose can then be used to perform a rigid body transformation on the original 3D feature image to obtain the projected 2D feature image.

[0072] For example, by processing the image data of the original two-dimensional feature image and the image data of the projected two-dimensional feature image, the relative pose between the original three-dimensional feature image and the original two-dimensional feature image can be obtained. Furthermore, the incremental pose and the initial pose of the original three-dimensional feature image can be fused to obtain a second global pose.

[0073] In this embodiment, the relative poses of the original 2D feature image and the projected 2D feature image are obtained through processing. Using this relative pose and an initial pose for the original 3D feature image, a second global pose for matching the original 2D feature image and the projected 2D feature image can be obtained. Thus, the relative pose improves the scale accuracy of image matching. Furthermore, obtaining the global pose using the relative pose and the initial pose accurately yields the global pose used for matching, thereby improving the accuracy of image matching. This global pose can be further fused with the first global pose to obtain the target global pose. Through multi-task processing, the scale accuracy of image matching and the matching accuracy of different modal data can be improved.

[0074] In one embodiment, determining the first global pose based on the projected two-dimensional feature image includes:

[0075] The projected two-dimensional feature image is input into the pre-trained target pose acquisition model so that the target pose acquisition model performs first pose acquisition processing on the projected two-dimensional feature image to obtain the first global pose.

[0076] Based on the original two-dimensional feature image and the projected two-dimensional feature image, the incremental pose is obtained, including:

[0077] The original two-dimensional feature image is input into the target pose acquisition model so that the pre-trained pose acquisition model can perform a second pose acquisition process on the original two-dimensional feature image and the projected two-dimensional feature image to obtain the incremental pose.

[0078] The target pose acquisition model refers to a data processing model capable of processing image data and acquiring pose from the processed image data. For example, it could be a neural network model that acquires pose through image data processing. The first pose acquisition process could be an image data processing method for acquiring the first global pose. The second pose acquisition process could be an image data processing method for acquiring the incremental pose.

[0079] For example, the target pose acquisition model obtains a first global pose by performing a first pose acquisition process on the image data of the projected two-dimensional feature image. The target pose acquisition model then performs a second pose acquisition process on both the original two-dimensional feature image and the projected two-dimensional feature image to obtain an incremental pose. Thus, the pose acquisition model can simultaneously perform multi-task processing to obtain both the first global pose and the incremental pose. This multi-task processing improves the scale accuracy of image matching, the matching accuracy of different modal data, and the efficiency of image matching. Furthermore, multi-task constraint learning can be performed during multi-task processing, thereby improving the accuracy of image matching.

[0080] In one embodiment, such as Figure 3 As shown, a pose acquisition model training method is provided, which includes the following steps:

[0081] S302, obtain the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample; the projected two-dimensional feature image of the sample is obtained by dimensionality reduction processing based on the three-dimensional feature image of the sample.

[0082] Here, the sample 3D feature image refers to the 3D feature image corresponding to the original 3D feature image, and the sample 3D feature image can be a sample 3D image with a linear topological structure. The original sample 2D feature image refers to the 2D feature image corresponding to the original 2D feature image, and the original sample 2D feature image can also be a sample 2D image with a linear topological structure. The sample projected 2D feature image refers to the sample 2D feature image obtained after dimensionality reduction processing of the sample 3D feature image. The sample 3D feature image, the original sample 2D feature image, and the sample projected 2D feature image can all be sample feature images used to train the pose acquisition model.

[0083] For example, using at least one image acquisition device, sample 3D feature images and original sample 2D feature images are acquired for training the pose acquisition model. By performing dimensionality reduction processing on the sample 3D feature images, sample projected 2D feature images corresponding to the sample 3D feature images can be obtained. The dimensionality reduction processing can be a rigid body transformation. Furthermore, the sample projected 2D feature images and the original sample 2D feature images can be used to train the pose acquisition model. For example, the sample projected 2D feature images and the original sample 2D feature images can be pre-matched feature images.

[0084] S304, input the original two-dimensional feature image of the sample and the two-dimensional feature image of the sample projection into the pose acquisition model to be trained, so that the pose acquisition model to be trained performs the first pose acquisition processing on the two-dimensional feature image of the sample projection to obtain the predicted first global pose; and performs the second pose acquisition processing on the original two-dimensional feature image of the sample and the two-dimensional feature image of the sample projection to obtain the predicted second global pose.

[0085] Here, predicting the first global pose refers to the first global pose predicted by the pose acquisition model. Predicting the second global pose refers to the second global pose predicted by the pose acquisition model.

[0086] For example, the pose acquisition model to be trained performs first pose acquisition processing on the sample projected 2D feature image to predict the first global pose; and performs second pose acquisition processing on the original sample 2D feature image and the sample projected 2D feature image to predict the second global pose. In this way, the pose acquisition model to be trained can predict based on the sample feature image, and can be trained based on the prediction result and the pre-acquired sample result (real result).

[0087] S306, Based on the predicted first global pose and the predicted second global pose, determine the loss value of the model for the pose to be trained.

[0088] S308, Based on the loss value of the pose acquisition model to be trained, the pose acquisition model to be trained is trained to obtain the target pose acquisition model.

[0089] Here, the loss value refers to the loss value used to train the pose acquisition model, which can be obtained from the prediction results and the actual results. For example, a loss value can be obtained by predicting the first global pose and the second global pose, and the pose acquisition model can be trained using this loss value to obtain a trained pose acquisition model. Alternatively, a first loss value can be determined by predicting the first global pose and the sample first global pose, and a second loss value can be determined by predicting the second global pose and the sample second global pose. The pose acquisition model can then be trained using both the first and second loss values ​​to obtain a trained pose acquisition model.

[0090] For example, based on the predicted first global pose and the predicted second global pose, a loss value can be determined for training the pose acquisition model, and the pose acquisition model can be trained using this loss value.

[0091] In this embodiment, the pose acquisition model is trained using pre-acquired sample poses and the predicted poses from the pose acquisition model to be trained, resulting in a trained pose acquisition model. This model can then be used to obtain a first global pose and predict a second global pose. By utilizing the pose acquisition model through multi-task processing, the accuracy of image matching scale and the matching accuracy of different modal data can be improved, thus resolving the problems of fuzzy matching scales and poor matching degrees between different modalities.

[0092] In one embodiment, such as Figure 4 As shown, based on the predicted first global pose and the predicted second global pose, the loss value for the pose acquisition model to be trained is determined, including:

[0093] S402, obtain the first global pose of the sample for the sample's three-dimensional feature image, and determine the first loss value based on the predicted first global pose and the sample's first global pose;

[0094] S404, obtain the second global pose of the sample for the sample's three-dimensional feature image and the sample's original two-dimensional feature image, and determine the second loss value based on the predicted second global pose and the sample's second global pose;

[0095] S406, Based on the first loss value and the second loss value, determine the loss value for the pose acquisition model to be trained.

[0096] Here, the first global pose of the sample refers to the global pose that can be directly obtained from the sample's 3D feature image. The second global pose of the sample refers to the global pose between the sample's 3D feature image and the original 2D feature image. It can be understood that both the first and second global poses are the true poses during model training, i.e., the actual results of model training. The first loss value can be obtained using a first loss function applied to the predicted first global pose and the sample's first global pose. The second loss value can be obtained using a second loss function applied to the predicted second global pose and the sample's second global pose.

[0097] For example, a first loss function is used to differentiate the predicted first global pose and the sample first global pose, resulting in a first loss value. A second loss function is then used to differentiate the second global pose and the sample second global pose, resulting in a second loss value. Further, the first and second loss values ​​can be used to determine the final loss value used to train the pose acquisition model.

[0098] In this embodiment, the pose acquisition model is trained by determining a first loss value and a second loss value. Multi-task training is performed, and the model is constrained by two loss values. Compared with single-task training using only one loss value, this application can achieve better training results, thereby obtaining a pose acquisition model with more accurate prediction results.

[0099] In one embodiment, determining the loss value for the pose acquisition model to be trained based on the predicted first global pose and the predicted second global pose includes:

[0100] Based on the predicted first global pose and the predicted second global pose, the predicted target global pose is obtained;

[0101] Based on the first global pose and the second global pose of the sample, the global pose of the target sample is obtained;

[0102] Based on the predicted global pose of the target and the global pose of the sample target, determine the loss value for the model to be trained based on the pose.

[0103] The prediction of the second global pose can be the prediction result obtained through the pose acquisition model.

[0104] For example, by predicting the first global pose and the second global pose, a fusion processing of the predicted target global pose is obtained. Further, a pre-defined loss function for the pose acquisition model can be used to process the predicted target global pose and the sample target global pose to obtain a loss value for the pose acquisition model. Thus, by determining the loss value using the final predicted target global pose and the actual target global pose, the training efficiency of the pose acquisition model can be improved.

[0105] In one embodiment, determining the loss value for the pose acquisition model to be trained based on the predicted first global pose, the predicted incremental pose, the sample first global pose, and the sample incremental pose further includes:

[0106] Obtain the initial pose of the sample based on the 3D feature image of the sample;

[0107] Based on the initial pose of the sample and the predicted incremental pose, the predicted second global pose is obtained;

[0108] Based on the predicted first global pose and the predicted second global pose, the predicted target global pose is obtained;

[0109] Based on the incremental pose and the initial pose of the sample, the second global pose of the sample is obtained;

[0110] Based on the first global pose and the second global pose of the sample, the global pose of the target sample is obtained;

[0111] Based on the predicted global pose of the target and the global pose of the sample target, determine the loss value for the model to be trained based on the pose.

[0112] The initial global pose of the sample can be obtained directly from the 3D feature image of the sample using a preset image acquisition device; that is, the sample pose corresponding to the 3D feature image of the sample. The predicted second global pose can be the prediction result obtained through a pose acquisition model. The predicted target global pose can be the prediction result obtained by fusing the predicted first global pose and the predicted second global pose. The second global pose of the sample refers to the sample pose obtained based on the incremental pose and the initial pose of the sample. The target global pose of the sample can be the sample pose obtained by fusing the first global pose and the second global pose of the sample (which can be used as the true result).

[0113] For example, the sample first global pose and sample second global pose are fused to obtain the sample target global pose. Then, the predicted target global pose is obtained by predicting the first global pose and the predicted second global pose. Further, a pre-defined loss function for the pose acquisition model can be used to process the predicted target global pose and the sample target global pose to obtain the loss value for the pose acquisition model. Thus, by determining the loss value using the final predicted target global pose and the actual target global pose, the training efficiency of the pose acquisition model can be improved.

[0114] In one embodiment, a second pose acquisition process is performed on the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample to obtain a predicted second global pose, including:

[0115] The original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample are processed to obtain the predicted incremental pose.

[0116] The initial pose of the original 3D feature image of the sample is obtained, and the predicted second global pose is obtained based on the initial pose and the predicted incremental pose.

[0117] For example, the initial pose can be obtained directly from the sample pose parameters of the sample's 3D feature image using a preset image acquisition device, i.e., the sample pose corresponding to the sample's 3D feature image. A second pose acquisition process is then performed on the original 2D feature image and the projected 2D feature image of the sample to obtain the predicted incremental pose. Through the fusion of the initial pose and the predicted incremental pose, a predicted second global pose can be obtained. Thus, by using the initial pose and the predicted incremental pose—that is, by using the initial pose and the predicted relative pose—to obtain the predicted second global pose, the pose acquisition model can be trained. This enables multi-task training of the pose acquisition model, thereby improving the image processing accuracy of the pose acquisition model.

[0118] In one embodiment, such as Figure 5 As shown, a pose acquisition model training method is provided, including the following steps:

[0119] S501: Using a preset image acquisition device, acquire the three-dimensional feature image of the sample and the original two-dimensional feature image of the sample, and perform rigid body transformation on the three-dimensional feature image of the sample to obtain the sample projection two-dimensional feature image corresponding to the three-dimensional feature image of the sample.

[0120] Here, the sample 3D feature image refers to the 3D feature image corresponding to the original 3D feature image, and the sample 3D feature image is a sample 3D image with a linear topological structure. The original sample 2D feature image refers to the 2D feature image corresponding to the original 2D feature image, and the original sample 2D feature image is a sample 2D image with a linear topological structure. The sample projected 2D feature image refers to the sample 2D feature image obtained after dimensionality reduction processing of the sample 3D feature image. The sample 3D feature image, the original sample 2D feature image, and the sample projected 2D feature image can all be sample feature images used to train the pose acquisition model.

[0121] For example, at least one image acquisition device is used to acquire sample three-dimensional feature images and sample original two-dimensional feature images for training the pose acquisition model. By performing rigid body transformation on the sample three-dimensional feature images, sample projected two-dimensional feature images corresponding to the sample three-dimensional feature images can be obtained. Furthermore, the pose acquisition model can be trained using the sample projected two-dimensional feature images and the sample original two-dimensional feature images.

[0122] S502, obtain the first global pose of the sample for the sample 3D feature image, and the incremental pose of the sample for the sample 3D feature image and the original 2D feature image of the sample.

[0123] Here, the first global pose of the sample refers to the first global pose of the sample that can be directly obtained from the sample's 3D feature image. The incremental pose of the sample refers to the relative pose between the sample's 3D feature image and the original 2D feature image of the sample. It can be understood that both the first global pose and the incremental pose of the sample are the true poses in the model training, that is, the true results of the model training.

[0124] For example, the model obtains the first global pose and the incremental pose of the sample for training the real results of the model. It can further obtain the prediction results obtained by the model through the first global pose, the incremental pose, and the pose. The model is then trained to obtain the trained model. The model can then obtain the first global pose and the incremental pose. This allows the model to obtain the target global pose through multiple tasks, improving the accuracy of the target global pose and thus improving the accuracy of image matching.

[0125] S503, input the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample into the pose acquisition model to be trained, so that the pose acquisition model to be trained can obtain the predicted first global pose based on the projected two-dimensional feature image of the sample, and obtain the predicted incremental pose based on the original two-dimensional feature image and the projected two-dimensional feature image.

[0126] Here, predicting the first global pose refers to the first global pose predicted by the pose acquisition model. Predicting the incremental pose refers to the incremental pose predicted by the pose acquisition model.

[0127] For example, the pose acquisition model to be trained predicts the first global pose by projecting the two-dimensional feature image of the sample; the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample are then predicted to obtain the incremental pose. In this way, the pose acquisition model to be trained can predict based on the sample feature image, and can be trained based on the prediction results and the pre-acquired sample results (real results).

[0128] S504. Determine the first loss value based on the predicted first global pose and the sample first global pose.

[0129] The first loss value can be obtained by using a first loss function for the predicted first global pose and the sample first global pose.

[0130] For example, the first loss function is used to differentiate the predicted first global pose and the sample first global pose to obtain the first loss value.

[0131] S505, determine the second loss value based on the predicted incremental pose and the sample incremental pose.

[0132] The second loss value can be obtained by using a second loss function for the predicted incremental pose and the sample incremental pose.

[0133] For example, the predicted incremental pose and the sample incremental pose are differentiated according to the second loss function to obtain the second loss value.

[0134] S506, Based on the first loss value and the second loss value, determine the loss value for the pose acquisition model to be trained.

[0135] For example, the loss value used to train the pose acquisition model can be determined based on the first loss value and the second loss value.

[0136] S507, based on the loss value of the pose acquisition model to be trained, train the pose acquisition model to be trained to obtain the target pose acquisition model.

[0137] For example, the pose acquisition model is trained based on the loss value determined by the first loss value and the second loss value to obtain the trained pose acquisition model. This allows for constrained training using two loss values, which, compared to single-task constrained training using only one loss value, yields better training results and a more accurate target pose acquisition model. Furthermore, the trained target pose acquisition model can be further utilized to obtain the first global pose and incremental pose, i.e., the absolute pose of the original 3D feature image and the relative pose between the original 3D and 2D feature images. This reduces image matching scale ambiguity and addresses the problem of low image matching accuracy across different modalities.

[0138] In one specific embodiment, a structural diagram of a method for matching three-dimensional medical images and two-dimensional medical images is shown below. Figure 6 As shown, the data flow includes the following:

[0139] The original 3D medical image 601 is projected to obtain a projected 2D medical image 603. This projected 2D medical image 603 is then input into an absolute pose acquisition model 604 to obtain an absolute pose 605. The original 2D medical image 602 and the projected 2D medical image 603 are input into a relative pose acquisition model 606 to obtain a relative pose 607. The relative pose 607, the absolute pose 605, and a pre-set, directly obtainable initial pose 608 are then fused to obtain the target global pose 609. Thus, by processing the original 3D and 2D medical images using a multi-task deep neural network model, relative and absolute poses can be obtained, and the optimal global pose can be output based on these poses. This overcomes the scale ambiguity problem associated with matching using relative poses and addresses the low accuracy issue caused by inconsistent modal data in absolute poses. Furthermore, the multi-task neural network prediction method, with its multiple constraints, improves the training effect and accuracy of the deep neural network model.

[0140] For example, the input data can be 3D blood vessel data after 3D image segmentation, 2D blood vessel data, and the initial pose of the 3D blood vessel data. The output can be the optimal pose of the 3D blood vessel data. A 2D blood vessel observation is simulated based on the initial pose of the 3D blood vessel data, and then fed into the absolute pose acquisition model 604 to directly output the absolute pose (3D global pose). Simultaneously, it is compared with the 2D blood vessel data input into the relative pose acquisition model 606 to obtain an incremental pose corresponding to the initial 3D pose. The initial 3D pose and the incremental pose can also yield the global 3D pose. Finally, the two global poses are fused to obtain the final pose.

[0141] In this embodiment, compared to the single loss function of traditional neural networks, the relative pose acquisition model provided in this application can have two loss functions: a loss function for absolute pose and a loss function for relative pose. The absolute pose can be simulated using the DRR (digitally reconstructed radiograph) algorithm at different camera angles. Simultaneously, the relative pose transformation relationship can be obtained from the training data. Thus, both loss functions can be trained simultaneously during training. During prediction, the network structure of the relative pose acquisition model outputs both absolute and relative poses. The initial pose of the relative pose acquisition model, combined with the relative pose (local pose increment) and the global pose, outputs another global pose, which is then fused with the current global pose of the absolute pose acquisition model.

[0142] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0143] Based on the same inventive concept, this application also provides an image matching device for implementing the image matching method and a pose acquisition model training device for implementing the pose acquisition model training method. The solution provided by the above-mentioned device is similar to the solution described in the above-mentioned method. Therefore, the specific limitations of one or more image matching devices and pose acquisition model training devices provided below can be found in the above-mentioned limitations of the image matching method and pose acquisition model training method, and will not be repeated here.

[0144] In one embodiment, such as Figure 7 As shown, an image matching device 700 is provided, including: a feature image acquisition module 710, a pose acquisition module 720, and a target pose acquisition module 730, wherein:

[0145] The feature image acquisition module 710 is used to acquire the original two-dimensional feature image to be matched and the projected two-dimensional feature image; the projected two-dimensional feature image is obtained by dimensionality reduction processing based on the original three-dimensional feature image to be matched.

[0146] The pose acquisition module 720 is used to determine a first global pose based on the projected two-dimensional feature image, and to determine a second global pose based on the original two-dimensional feature image and the projected two-dimensional feature image.

[0147] The target pose acquisition module 730 is used to determine the target global pose for the original three-dimensional feature image and the original two-dimensional feature image based on the first global pose and the second global pose; the target global pose is used to match the original three-dimensional feature image and the original two-dimensional feature image.

[0148] In one embodiment, the pose acquisition module includes an incremental pose unit and an initial pose unit.

[0149] The incremental pose unit is used to obtain the incremental pose for the original 3D feature image and the original 2D feature image based on the original 2D feature image and the projected 2D feature image. The initial pose unit is used to obtain the initial pose for the original 3D feature image, and obtain the second global pose based on the initial pose and the incremental pose.

[0150] In one embodiment, the pose acquisition module is used to obtain an incremental pose based on the original two-dimensional feature image and the projected two-dimensional feature image; obtain an initial pose for the original three-dimensional feature image; and obtain a second global pose based on the initial pose and the incremental pose.

[0151] In one embodiment, the pose acquisition module is used to input the projected two-dimensional feature image into the pre-trained target pose acquisition model, so that the target pose acquisition model performs a first pose acquisition process on the projected two-dimensional feature image to obtain a first global pose; it is also used to input the original two-dimensional feature image into the target pose acquisition model, so that the target pose acquisition model performs a second pose acquisition process on the original two-dimensional feature image and the projected two-dimensional feature image to obtain an incremental pose.

[0152] In one embodiment, such as Figure 8 As shown, a pose acquisition model training device 800 is provided, including: a sample image acquisition module 810, a predicted pose acquisition module 820, a loss value determination module 830, and a model training module 840.

[0153] The sample image acquisition module 810 is used to acquire the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample; the projected two-dimensional feature image of the sample is obtained by dimensionality reduction processing based on the three-dimensional feature image of the sample.

[0154] The pose acquisition module 820 is used to input the original two-dimensional feature image of the sample and the two-dimensional feature image of the sample projection into the pose acquisition model to be trained, so that the pose acquisition model to be trained performs a first pose acquisition process on the two-dimensional feature image of the sample projection to obtain a predicted first global pose; and performs a second pose acquisition process on the original two-dimensional feature image of the sample and the two-dimensional feature image of the sample projection to obtain a predicted second global pose.

[0155] The loss value determination module 830 is used to determine the loss value of the model for the pose to be trained based on the predicted first global pose and the predicted second global pose.

[0156] The model training module 840 is used to train the pose acquisition model to be trained based on the loss value of the pose acquisition model to be trained, so as to obtain the target pose acquisition model.

[0157] In one embodiment, the loss value determination module includes a first loss value determination unit, a second loss value determination unit, and a target loss value determination unit.

[0158] The first loss value determination unit is used to acquire the first global pose of the sample 3D feature image and determine the first loss value based on the predicted first global pose and the sample's first global pose. The second loss value determination unit is used to acquire the second global pose of the sample 3D feature image and the original 2D feature image and determine the second loss value based on the predicted second global pose and the sample's second global pose. The target loss value determination unit is used to determine the loss value of the model for the pose to be trained based on the first loss value and the second loss value.

[0159] In one embodiment, the loss value determination module includes a prediction pose unit and a sample pose unit.

[0160] The pose prediction unit is used to obtain the predicted global pose of the target based on the predicted first global pose and the predicted second global pose. The sample pose unit is used to obtain the sample target global pose based on the sample first global pose and the sample second global pose. The target loss determination unit is used to determine the loss value of the pose acquisition model for the target to be trained based on the predicted target global pose and the sample target global pose.

[0161] In one embodiment, the predicted pose acquisition module includes an incremental pose unit and a second global pose unit.

[0162] The incremental pose unit is used to perform second pose acquisition processing on the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample to obtain the predicted incremental pose.

[0163] The second global pose unit is used to obtain the initial pose of the original 3D feature image of the sample, and to obtain the predicted second global pose based on the initial pose and the predicted incremental pose.

[0164] Each module in the aforementioned image matching device and pose acquisition model training device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0165] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores raw 3D feature images, raw 2D feature images, sample 3D feature images, and sample raw 2D feature maps. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements an image matching method and a pose acquisition model training method.

[0166] Those skilled in the art will understand that Figure 9 The structures shown are merely block images of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figures, or combine certain components, or have different component arrangements.

[0167] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0168] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0169] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0170] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0171] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image matching method, characterized in that, The method includes: Obtain the original two-dimensional feature image to be matched and the projected two-dimensional feature image; the projected two-dimensional feature image is obtained by dimensionality reduction processing based on the original three-dimensional feature image to be matched. Based on the projected two-dimensional feature image, a first global pose is determined, and based on the original two-dimensional feature image and the projected two-dimensional feature image, an incremental pose is obtained, and based on the initial pose of the original three-dimensional feature image and the incremental pose, a second global pose is obtained. Based on the first global pose and the second global pose, a target global pose is determined for the original 3D feature image and the original 2D feature image; the target global pose is used to match the original 3D feature image and the original 2D feature image.

2. The method according to claim 1, characterized in that, Before obtaining the second global pose based on the initial pose of the original 3D feature image and the incremental pose, the method further includes: Obtain the initial pose for the original 3D feature image.

3. The method according to claim 2, characterized in that, The step of determining the first global pose based on the projected two-dimensional feature image includes: The projected two-dimensional feature image is input into a pre-trained target pose acquisition model so that the target pose acquisition model performs first pose acquisition processing on the projected two-dimensional feature image to obtain the first global pose. The step of obtaining the incremental pose based on the original two-dimensional feature image and the projected two-dimensional feature image includes: The original two-dimensional feature image is input into the target pose acquisition model, so that the target pose acquisition model performs a second pose acquisition process on the original two-dimensional feature image and the projected two-dimensional feature image to obtain the incremental pose.

4. A method for training a pose acquisition model, characterized in that, The method includes: The original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample are obtained; the projected two-dimensional feature image of the sample is obtained by dimensionality reduction processing based on the three-dimensional feature image of the sample. The original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample are input into the pose acquisition model to be trained, so that the pose acquisition model to be trained performs a first pose acquisition process on the projected two-dimensional feature image of the sample to obtain a predicted first global pose; and performs a second pose acquisition process on the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample to obtain a predicted incremental pose, and obtains a predicted second global pose based on the initial pose of the three-dimensional feature image of the sample and the predicted incremental pose. Based on the predicted first global pose and the predicted second global pose, determine the loss value of the model for the pose to be trained; Based on the loss value of the pose acquisition model to be trained, the pose acquisition model to be trained is trained to obtain the target pose acquisition model.

5. The method according to claim 4, characterized in that, The step of determining the loss value for the pose acquisition model to be trained based on the predicted first global pose and the predicted second global pose includes: Obtain the first global pose of the sample for the three-dimensional feature image of the sample, and determine the first loss value based on the predicted first global pose and the first global pose of the sample. Obtain the second global pose of the sample for the three-dimensional feature image of the sample and the original two-dimensional feature image of the sample, and determine the second loss value based on the predicted second global pose and the second global pose of the sample; Based on the first loss value and the second loss value, a loss value is determined for the pose acquisition model to be trained.

6. The method according to claim 5, characterized in that, The step of determining the loss value for the pose acquisition model to be trained based on the predicted first global pose and the predicted second global pose includes: Based on the predicted first global pose and the predicted second global pose, the predicted target global pose is obtained; Based on the first global pose of the sample and the second global pose of the sample, the global pose of the target sample is obtained; Based on the predicted global pose of the target and the sample global pose of the target, determine the loss value for the pose acquisition model to be trained.

7. The method according to claim 4, characterized in that, Before obtaining the predicted second global pose based on the initial pose of the sample 3D feature image and the predicted incremental pose, the method further includes: Obtain the initial pose of the three-dimensional feature image of the sample.

8. An image matching device, characterized in that, The device includes: The feature image acquisition module is used to acquire the original two-dimensional feature image to be matched and the projected two-dimensional feature image; the projected two-dimensional feature image is obtained by dimensionality reduction processing based on the original three-dimensional feature image to be matched. The pose acquisition module is used to determine a first global pose based on the projected two-dimensional feature image, and to obtain an incremental pose based on the original two-dimensional feature image and the projected two-dimensional feature image, and to obtain a second global pose based on the initial pose of the original three-dimensional feature image and the incremental pose. The target pose acquisition module is used to determine the target global pose for the original three-dimensional feature image and the original two-dimensional feature image based on the first global pose and the second global pose; the target global pose is used to match the original three-dimensional feature image and the original two-dimensional feature image.

9. A pose acquisition model training device, characterized in that, The device includes: The sample image acquisition module is used to acquire the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample; the projected two-dimensional feature image of the sample is obtained by dimensionality reduction processing based on the three-dimensional feature image of the sample. The pose prediction module is used to input the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample into the pose acquisition model to be trained, so that the pose acquisition model to be trained performs a first pose acquisition process on the projected two-dimensional feature image of the sample to obtain a predicted first global pose; and performs a second pose acquisition process on the original two-dimensional feature image of the sample and the projected two-dimensional feature image of the sample to obtain a predicted incremental pose, and obtains a predicted second global pose based on the initial pose of the three-dimensional feature image of the sample and the predicted incremental pose. The loss value determination module is used to determine the loss value of the pose acquisition model for the pose to be trained based on the predicted first global pose and the predicted second global pose. The model training module is used to train the pose acquisition model to be trained based on the loss value of the pose acquisition model to be trained, so as to obtain the target pose acquisition model.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images