Image matching method, device and computer equipment

Abstract

The present application relates to an image matching method, apparatus, and computer equipment. The method comprises: obtaining a projected two-dimensional matching point set and an original two-dimensional matching point set; wherein the projected two-dimensional matching point set is obtained by performing dimensionality reduction processing on the three-dimensional matching point set corresponding to the three-dimensional feature image to be matched; the original two-dimensional matching point set is obtained based on the two-dimensional feature image to be matched; obtaining the respective similarities between each projected two-dimensional matching point in the projected two-dimensional matching point set and the original two-dimensional matching point set, and superimposing the respective similarities to obtain a target similarity; performing gradient descent processing on the target similarity, and obtaining target relative pose parameters when the target similarity meets a first preset condition; and matching the three-dimensional feature image with the original two-dimensional feature image using the target relative pose parameters. This method can improve the accuracy of image matching.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image matching method, apparatus, and computer device. Background Art

[0002] Image registration involves transforming images from the same or different imaging modalities to align their spatial positions and coordinates. The result of registration is that matching points, or at least all meaningful points, on the two images are matched. However, due to the varying imaging principles of different devices, images of multiple modalities often require integration of information from multiple images for analysis and processing. The primary challenge in integrating multiple image information is strict image alignment, or image registration.

[0003] Existing image matching technologies use sampling-based algorithms such as the Powell algorithm or the Nelder-Mead (Downhill Simplex Method) algorithm to perform image matching. However, the matching process suffers from a problem of low matching accuracy. Summary of the Invention

[0004] Based on this, it is necessary to provide an image matching method, device and computer equipment that can improve accuracy in response to the above technical problems.

[0005] In a first aspect, the present application provides an image matching method, the method comprising:

[0006] Obtaining a projected two-dimensional matching point set and an original two-dimensional matching point set; wherein the projected two-dimensional matching point set is obtained by performing dimensionality reduction processing on the three-dimensional matching point set corresponding to the three-dimensional feature image to be matched; and the original two-dimensional matching point set is obtained based on the two-dimensional feature image to be matched;

[0007] Obtaining the similarities between each projected two-dimensional matching point in the projected two-dimensional matching point set and the original two-dimensional matching point set, and superimposing each similarity to obtain the target similarity;

[0008] Performing gradient descent processing on the target similarity, and obtaining the target relative pose parameters when the target similarity meets the first preset condition;

[0009] The three-dimensional feature image and the original two-dimensional feature image are matched through the target relative pose parameters.

[0010] In one embodiment, obtaining the similarities between each projected two-dimensional matching point in the projected two-dimensional matching point set and the original two-dimensional matching point set includes:

[0011] Obtain each projected two-dimensional matching point in the projected two-dimensional matching point set and each original two-dimensional matching point in the original two-dimensional matching point set;

[0012] Obtain the distance parameters between each projected two-dimensional matching point and each original two-dimensional matching point;

[0013] According to each distance parameter between any projected two-dimensional matching point and each original two-dimensional matching point, the similarity between any projected two-dimensional matching point and the set of original two-dimensional matching points is obtained.

[0014] In one embodiment, obtaining the similarity between any projected two-dimensional matching point and the set of original two-dimensional matching points based on the distance parameters between any projected two-dimensional matching point and the original two-dimensional matching points includes:

[0015] Get the preset hyperparameters;

[0016] According to the ratio of each distance parameter to the preset hyperparameter, the similarity between any projected two-dimensional matching point and each original two-dimensional matching point is obtained;

[0017] The similarity between any projected two-dimensional matching point and the original two-dimensional matching point set is obtained based on the sum of the similarities of each matching point.

[0018] In one embodiment, performing gradient descent processing on the target similarity includes:

[0019] Use Gaussian kernel function to characterize the similarity of matching points;

[0020] The target similarity is processed by gradient descent using the Gaussian kernel function.

[0021] In one embodiment, a Gaussian kernel function is used to perform gradient descent processing on the target similarity, including:

[0022] The exponential function in the Gaussian kernel function is used as the numerator and the known two-norm is used as the denominator to obtain the iterative weight parameter; the known two-norm is obtained based on the projected two-dimensional matching points, the original two-dimensional matching points and the preset relative pose parameters;

[0023] Get the unknown two-norm, multiply the iterative weight parameter by the unknown two-norm to get the gradient descent function;

[0024] According to the gradient descent function, the target similarity is processed by second-order gradient descent.

[0025] In one embodiment, obtaining the target relative pose parameters includes:

[0026] When the gradient descent function satisfies the second preset condition, the target relative pose parameters are obtained.

[0027] In one embodiment, a second-order gradient descent process is performed on the target similarity according to a gradient descent function, including:

[0028] According to the gradient descent function, the target similarity is processed by second-order gradient descent in the special Euclidean group;

[0029] In one embodiment, obtaining target relative pose parameters includes:

[0030] In the special Euclidean group, the target relative pose matrix is ​​obtained and used as the target relative pose parameter.

[0031] In a second aspect, the present application further provides an image matching device, comprising:

[0032] A matching point acquisition module is used to project a two-dimensional matching point set and an original two-dimensional matching point set; wherein the projected two-dimensional matching point set is obtained by performing dimensionality reduction processing on the three-dimensional matching point set corresponding to the three-dimensional feature image to be matched; and the original two-dimensional matching point set is obtained based on the two-dimensional feature image to be matched;

[0033] A similarity acquisition module is used to obtain the similarities between each projected two-dimensional matching point in the projected two-dimensional matching point set and the original two-dimensional matching point set, and to superimpose the similarities to obtain the target similarity;

[0034] A target posture acquisition module is used to perform gradient descent processing on the target similarity and obtain the target relative posture parameters when the target similarity meets the first preset condition;

[0035] The matching module matches the three-dimensional feature image and the original two-dimensional feature image through the target relative pose parameters.

[0036] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0037] The above-mentioned image matching method, device and computer equipment obtain a projected two-dimensional matching point set and an original two-dimensional matching point set; wherein, the projected two-dimensional matching point set is obtained by dimensionality reduction processing based on the three-dimensional matching point set corresponding to the three-dimensional feature image to be matched; the original two-dimensional matching point set is obtained based on the two-dimensional feature image to be matched; the similarities between each projected two-dimensional matching point in the projected two-dimensional matching point set and the original two-dimensional matching point set are obtained, and each similarity is superimposed to obtain the target similarity; the target similarity is gradient descent processed, and when the target similarity meets the first preset condition, the target relative posture parameter is obtained; the three-dimensional feature image and the original two-dimensional feature image are matched through the target relative posture parameter. By matching each projected two-dimensional matching point in the projected two-dimensional matching point set with the original two-dimensional matching point set, the target similarity is obtained, and by performing gradient descent processing on the target similarity, the target relative posture parameters can be obtained through gradient descent processing to complete the matching of the three-dimensional feature image and the original two-dimensional feature image. Compared with the traditional matching method, the present application matches each projected two-dimensional matching point with the original two-dimensional matching point set, which is equivalent to matching each matching point with the center line formed by the original two-dimensional matching point set, rather than using point-to-point matching to obtain similarity, thereby improving the accuracy of the target relative posture parameters and thus improving the accuracy of image matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A diagram showing an application environment of an image matching method in one embodiment;

[0039] Figure 2 1 is a flow chart of an image matching method according to an embodiment;

[0040] Figure 3 is a schematic flow chart of an image matching method in another embodiment;

[0041] Figure 4 1 is a flow chart of an image matching method in another embodiment;

[0042] Figure 5 is a structural block diagram of an image matching device in one embodiment;

[0043] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0045] The image matching method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. The data storage system can store three-dimensional feature images and two-dimensional feature images. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, Internet of Things devices, and portable wearable devices. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0046] In one embodiment, Figure 2 As shown, an image matching method is provided, which is applied to Figure 1 The following steps are used as an example to illustrate the server in the example:

[0047] S202, obtaining a projected two-dimensional matching point set and an original two-dimensional matching point set; wherein the projected two-dimensional matching point set is obtained by dimensionality reduction processing based on the three-dimensional matching point set corresponding to the three-dimensional feature image to be matched; the original two-dimensional matching point set is obtained based on the two-dimensional feature image to be matched.

[0048] The three-dimensional feature image may be a three-dimensional image with a linear topological structure; a three-dimensional image with a linear topological structure means an image exhibiting a linear topological relationship, for example, a three-dimensional image that displays a linear topological relationship, such as a three-dimensional medical image displaying human blood vessels or tissue. The two-dimensional feature image may be a two-dimensional image with a linear topological structure, for example, a two-dimensional medical image displaying human blood vessels or tissue. The three-dimensional matching point set may be a set of nodes in the three-dimensional feature image, for example, the three-dimensional matching point set may include each node in the three-dimensional feature image. The original two-dimensional matching point set may be a set of nodes in the two-dimensional feature image, for example, the original two-dimensional matching point set may include each node in the two-dimensional feature image. The three-dimensional matching points may be 3D skeleton points in a three-dimensional medical image, and the original two-dimensional matching points may be 2D skeleton points in a two-dimensional medical image.

[0049] Exemplarily, a three-dimensional feature image to be matched and a two-dimensional feature image to be matched are obtained, for example, a preoperative three-dimensional medical image and an intraoperative two-dimensional medical image are obtained. By matching the preoperative three-dimensional medical image and the intraoperative two-dimensional medical image, the preoperative medical data has a corresponding relationship in the intraoperative application process, which is beneficial to the progress of the operation.

[0050] A three-dimensional matching point set corresponding to the three-dimensional feature image and an original two-dimensional matching point set corresponding to the two-dimensional feature image are obtained. By matching the matching points in the three-dimensional matching point set and the matching points in the original two-dimensional matching point set, parameters for matching the two-dimensional feature image and the three-dimensional feature image can be obtained to complete the matching of the two-dimensional feature image and the three-dimensional feature image. For example, a three-dimensional medical image with a tubular topology can be subjected to centerline processing to obtain a three-dimensional medical image containing a centerline structure. The centerline can be discretized to obtain a three-dimensional medical image containing multiple three-dimensional discrete nodes. The multiple three-dimensional discrete nodes are formed into a three-dimensional matching point set. At the same time, the above-mentioned processing for three-dimensional medical images can be performed on a two-dimensional medical image with a tubular topology to obtain a two-dimensional medical image with multiple two-dimensional discrete nodes. The multiple two-dimensional discrete nodes are formed into an original two-dimensional matching point set.

[0051] Exemplarily, a dimensionality reduction process is performed on a three-dimensional feature image containing a set of three-dimensional matching points to obtain a projected two-dimensional feature image, and a projected two-dimensional matching point set is obtained from the projected two-dimensional feature image. For example, a rigid body transformation can be performed on the three-dimensional feature image to obtain the projected two-dimensional feature image. In this way, by performing a rigid body transformation on the three-dimensional matching point set, a projected two-dimensional matching point set can be obtained, and matching can be performed by using the projected two-dimensional matching point set and each match in the original two-dimensional matching point set. That is, the present application performs matching in the two-dimensional dimension, avoiding the problem of cross-modal matching when one is in the three-dimensional dimension and the other is in the two-dimensional dimension, which easily leads to inaccurate matching.

[0052] For example, a dimensionality reduction process is performed on a 3D matching point set using preset relative pose parameters to obtain a projected 2D matching point set. The relative pose parameters may be parameters used to rotate and translate a 3D feature image. For example, the 3D feature image may be rotated and translated using the relative pose parameters to obtain a corresponding projected 2D feature image. As an example, the relative pose parameters may be used to perform dimensionality reduction on a 3D feature image containing a 3D matching point set to obtain a projected 2D feature image, and the projected 2D matching point set may be obtained from the projected 2D feature image.

[0053] S204 , obtaining respective similarities between each projected two-dimensional matching point in the projected two-dimensional matching point set and the original two-dimensional matching point set, and superimposing the respective similarities to obtain a target similarity.

[0054] The projected 2D matching points refer to the nodes in the projected 2D matching point set, and the similarity refers to the degree of matching between the projected 2D matching points and the original 2D matching point set.

[0055] Exemplarily, based on the respective similarities between each projected 2D matching point and the original 2D matching point set, the respective similarities are superimposed to obtain a target similarity used to characterize the degree of matching. The target similarity can be characterized using a similarity function. For example, based on the similarity between each projected 2D matching point and the center line formed by each original 2D matching point in the original 2D matching point set, the respective similarities corresponding to each projected 2D matching point can be obtained, and each similarity can be represented by a function. For example, each similarity can be characterized by using a kernel function. That is, each projected 2D matching point and the original 2D matching point set are substituted into the kernel function, and the kernel function can be used to characterize the similarity between the projected 2D matching point and the original 2D matching point set. The respective similarities are superimposed, i.e., the kernel function is superimposed. In this way, by matching each projected 2D matching point with the original 2D matching point set, it is equivalent to matching each projected 2D matching point with the center line of the 2D feature image, i.e., matching between points and lines. Compared with point-to-point matching, the matching accuracy can be improved.

[0056] S206 , performing gradient descent processing on the target similarity, and obtaining target relative pose parameters when the target similarity meets a first preset condition.

[0057] The first preset condition may be a condition corresponding to maximizing the similarity, or a condition corresponding to obtaining a similarity within a preset range. The gradient descent may be a first-order optimization algorithm, or a processing method that performs iterative search using a specified step size.

[0058] S208 , matching the three-dimensional feature image and the original two-dimensional feature image using the target relative posture parameters.

[0059] For example, the target similarity can be represented by a kernel function, and a gradient descent process is performed on the kernel function used to represent the similarity. When the kernel function satisfies a preset condition, the target relative pose parameters are obtained. For example, when the kernel function is minimized or maximized, the unknown relative pose parameters in the kernel function can be determined as known relative pose parameters, and the relative pose parameters are used as the target relative pose parameters.

[0060] The target relative pose parameters can be used to match the 3D feature image with the original 2D feature image. For example, the target relative pose parameters can be used to match preoperative 3D medical images with intraoperative 2D medical images.

[0061] In this embodiment, the target similarity is obtained by matching each projected two-dimensional matching point in the projected two-dimensional matching point set with the original two-dimensional matching point set, and the target similarity is subjected to gradient descent processing. The target relative posture parameters can be obtained through gradient descent processing to complete the matching of the three-dimensional feature image and the original two-dimensional feature image. Compared with the traditional matching method, the present application matches each projected two-dimensional matching point with the original two-dimensional matching point set, which is equivalent to matching each matching point with the center line formed by the original two-dimensional matching point set. Compared with obtaining the similarity by matching between points, the accuracy of the target relative posture parameters can be improved, thereby improving the accuracy of image matching.

[0062] In one embodiment, Figure 3 As shown, obtaining the similarities between each projected two-dimensional matching point in the projected two-dimensional matching point set and the original two-dimensional matching point set includes:

[0063] S302, obtaining each projected two-dimensional matching point in the projected two-dimensional matching point set and each original two-dimensional matching point in the original two-dimensional matching point set;

[0064] S304, obtaining distance parameters between each projected two-dimensional matching point and each original two-dimensional matching point;

[0065] S306 , obtaining the similarity between any projected two-dimensional matching point and the set of original two-dimensional matching points based on the distance parameters between any projected two-dimensional matching point and the original two-dimensional matching points.

[0066] The projected 2D matching point refers to a node in the projected 2D matching point set. The original 2D matching point refers to a node in the original 2D matching point set. The distance parameter represents the distance between the projected 2D matching point and the original 2D matching point, for example, the Euclidean distance. The distance parameter can be represented by a distance function, for example, the Euclidean distance in a Gaussian kernel function.

[0067] For example, the distance between the projected 2D matching point and the original 2D matching point can be represented by the Euclidean distance. Based on the Euclidean distances between any projected 2D matching point and each original 2D matching point, the Euclidean distances are superimposed to obtain the similarity between the projected 2D matching point and the set of original 2D matching points. For example, the Euclidean distance can be used to establish a kernel function for representing similarity, such as a Gaussian kernel function.

[0068] In this embodiment, the similarity between the projected two-dimensional matching point and the set of original two-dimensional matching points is obtained by calculating the distance parameters between the projected two-dimensional matching point and the original two-dimensional matching point set. The matching degree can be determined based on the distance parameters, thereby obtaining accurate matching parameters and improving the accuracy of image matching.

[0069] In one embodiment, Figure 4 As shown, according to the distance parameters between any projected 2D matching point and each original 2D matching point, the similarity between any projected 2D matching point and the set of original 2D matching points is obtained, including:

[0070] S402, obtaining preset hyperparameters;

[0071] S404, obtaining the similarity between any projected two-dimensional matching point and each original two-dimensional matching point based on the ratio of each distance parameter to the preset hyperparameter;

[0072] S406 , obtaining the similarity between any projected two-dimensional matching point and the original two-dimensional matching point set based on the sum of the similarities of the matching points.

[0073] Here, hyperparameters refer to pre-set parameters that can be obtained without relying on training data. Matching point similarity refers to the similarity between the projected 2D matching points and the original 2D matching points.

[0074] For example, using preset hyperparameters, each distance parameter is subjected to matching scaling. Based on the ratio of each distance parameter to the preset hyperparameter, the similarity between the projected 2D matching point and the original 2D matching point can be obtained. By adjusting the size of the hyperparameters, the matching accuracy can be adjusted, achieving a "coarse to fine" matching accuracy. By superimposing the similarities of each matching point, the similarity between the projected 2D matching point and the centerline of the original 2D matching point set can be obtained, that is, the match between the matching point and the centerline.

[0075] In this embodiment, by utilizing hyperparameters to determine the similarity of matching points, the matching accuracy can be controlled during the matching process, thereby achieving "coarse to fine" matching accuracy and meeting various matching accuracy requirements.

[0076] In one embodiment, performing gradient descent processing on target similarity includes:

[0077] Use Gaussian kernel function to characterize the similarity of matching points;

[0078] The target similarity is processed by gradient descent using the Gaussian kernel function.

[0079] For example, a Gaussian kernel function can be used to perform gradient descent processing on target similarity. This gradient descent processing can be understood as increasing the similarity between each projected two-dimensional matching point and each original two-dimensional matching point. Furthermore, when the similarity is maximized, the corresponding target relative pose parameters can be obtained. For example, the Gaussian kernel function can be processed using a first-order gradient method, and the target relative pose parameters can be obtained when the Gaussian kernel function is minimized. In this way, using a Gaussian kernel function to represent similarity can reduce the amount of computation, thereby improving the efficiency of image matching.

[0080] In one embodiment, a Gaussian kernel function is used to perform gradient descent processing on the target similarity, including:

[0081] The exponential function in the Gaussian kernel function is used as the numerator and the known two-norm is used as the denominator to obtain the iterative weight parameter; the known two-norm is obtained based on the projected two-dimensional matching points, the original two-dimensional matching points and the preset relative pose parameters;

[0082] Get the unknown two-norm, multiply the iterative weight parameter by the unknown two-norm to get the gradient descent function;

[0083] According to the gradient descent function, the target similarity is processed by second-order gradient descent.

[0084] Among them, the unknown two-norm may include unknown relative pose parameters.

[0085] For example, an exponential function containing the distance between the projected two-dimensional matching point and the original two-dimensional matching point can be used as the numerator, and a known second norm can be used as the denominator to obtain an iterative weight parameter. The iterative weight parameter is multiplied by the unknown second norm to obtain a gradient descent function, which can be processed using a second-order gradient descent method. When the gradient descent function is minimized, the target relative pose parameters are obtained. In this way, the Gaussian kernel function containing an exponential function can be converted into a gradient descent function without an exponential function, which can be solved using a second-order gradient method, thereby increasing the solution speed and thus improving the efficiency of image matching.

[0086] In one embodiment, obtaining target relative pose parameters includes:

[0087] When the gradient descent function satisfies the second preset condition, the target relative pose parameters are obtained.

[0088] The second preset condition may be a condition corresponding to minimization of the gradient descent function, or a condition corresponding to convergence of the gradient descent function.

[0089] For example, the target relative pose parameters can be obtained when the gradient descent function is minimized. Thus, when the gradient descent function satisfies the second preset condition, obtaining the target relative pose parameters can accurately obtain the target relative pose parameters, improve the accuracy of obtaining the target relative pose parameters, and improve the accuracy of image matching.

[0090] In one embodiment, a second-order gradient descent process is performed on target similarity according to a gradient descent function, including:

[0091] According to the gradient descent function, a second-order gradient descent process is performed on the target similarity in a special Euclidean group.

[0092] In one embodiment, obtaining target relative pose parameters includes:

[0093] In the special Euclidean group, the target relative pose matrix is ​​obtained and used as the target relative pose parameter.

[0094] Here, the special Euclidean group refers to a group that meets certain requirements, for example, a Lie group, which has the continuous property and can be the SE(3) special Euclidean group.

[0095] For example, relative pose parameters can be represented by matrices in a special Euclidean group, and second-order gradient descent processing can be performed on the gradient descent function on the special Euclidean group. Performing second-order gradient descent processing on the special Euclidean group can increase the processing speed of the second-order gradient descent processing, thereby improving the efficiency of obtaining relative pose parameters and, in turn, improving the efficiency of image matching.

[0096] In one embodiment, the dimensionality reduction process is performed on the three-dimensional matching point set using the preset relative pose parameters to obtain a projected two-dimensional matching point set, including:

[0097] Performing coordinate transformation on the three-dimensional matching point set with respect to the coordinate system carried by the imaging device to obtain the three-dimensional matching point set after coordinate transformation; the imaging device is used to obtain the three-dimensional matching points;

[0098] Using the preset relative pose parameters, the dimensionality reduction process is performed on the three-dimensional matching point set after coordinate transformation to obtain the projected two-dimensional matching point set.

[0099] For example, the coordinate system of the imaging device is transformed to obtain a transformed set of three-dimensional matching points. Using preset relative pose parameters, this transformed set of three-dimensional matching points is then subjected to dimensionality reduction to obtain a projected two-dimensional matching point set. In this way, transforming the coordinate system of the imaging device unifies the coordinate systems corresponding to the three-dimensional feature image and the two-dimensional feature image, thereby improving the accuracy of image matching.

[0100] In one embodiment, performing coordinate transformation on the three-dimensional matching point set with respect to the coordinate system carried by the imaging device to obtain the three-dimensional matching point set after coordinate transformation includes:

[0101] Get the center of the initial coordinate system for the imaging device and the transformation matrix for the initial coordinate system;

[0102] Each three-dimensional matching point in the three-dimensional matching point set is transformed using the center of the initial coordinate system and the transformation matrix to obtain a three-dimensional matching point set after coordinate transformation.

[0103] For example, each 3D matching point in the set of 3D matching points is transformed based on the center of the imaging device's initial coordinate system and the transformation matrix to obtain a transformed 3D matching point set. In this way, by performing the coordinate system transformation based on the center of the imaging device's coordinate system, the transformed coordinate system can be accurately obtained, thereby improving the accuracy of image matching.

[0104] In a specific embodiment, a matching method for preoperative 3D medical images and intraoperative 2D medical images is provided. For example, the relative position of the target can be obtained by the following formula (1):

[0105]

[0106] Where Ω3 is a set of three-dimensional matching points (such as a set of 3D skeleton points), Ω2 is a set of two-dimensional matching points (such as a set of 2D skeleton points), and π() is a projection from three dimensions to two dimensions. c∈R 3 is the center of the DSA device, which is a device that acquires three-dimensional medical images. l is a hyperparameter. w i is the weight for 3D medical images. dsa_ is the transformation matrix from the known CB coordinate system to the DSA device camera coordinate system. ΔT is the rotation and translation of the 3D matching point to be determined, i.e., the relative pose.

[0107] Formula (1) is difficult to express in standard least squares form, and therefore cannot be solved using the traditional Gauss-Newton second-order gradient method. This application provides a solution optimization method, comprising the following steps:

[0108] remember

[0109] but

[0110] make but

[0111] Use the preset relative pose parameter ΔT to bring it into w, so that the weight is continuously changed during the optimization process The second-order gradient method can be used to continuously optimize the φ function in formula (5) to achieve the purpose of second-order search and improve the speed of gradient descent processing.

[0112] For example, due to contrast agents and imaging mechanisms, matching points in both 3D images (such as 3D skeleton points) and 2D images (such as 2D skeleton points) may be missing or redundant, leading to anomalies during matching. Mainstream algorithms for skeleton graph matching optimization use sampling-based algorithms such as the Powell algorithm or the Nelder-Mead algorithm, which are not suitable for scenarios with high real-time requirements.

[0113] The current matching algorithm matches the 3D reprojection point with the single nearest 2D skeleton point, or determines the relative pose based on the minimum distance between the 3D reprojection point and the single nearest 2D skeleton point. The above methods all match a single 3D reprojection point with a single 2D skeleton point, and all have the problem of low accuracy. The matching method for preoperative three-dimensional medical images and intraoperative two-dimensional medical images provided in this application can use Gaussian kernel functions for non-parametric modeling in the matching process of 3D skeleton points and 2D skeleton points. The non-parametric model can perform "coarse to fine" matching of matching at different scales. The above non-parametric model is solved by iterative reweighted least squares method. It can be quickly derived by second-order descent. Using a special Euclidean group (SE3) to express the rigid body transformation (i.e., rotation and translation) of 3D skeleton points (such as 3D vascular data) can further improve the convergence speed of the algorithm iteration.

[0114] In this embodiment, the Gaussian kernel function is used for modeling, so that each three-dimensional matching point can be matched with each two-dimensional matching point, which is equivalent to matching the three-dimensional matching point with the center line contained in the two-dimensional medical image, that is, matching between points and lines. Compared with point-to-point matching, the matching accuracy can be improved.

[0115] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0116] Based on the same inventive concept, embodiments of the present application also provide an image matching device for implementing the aforementioned image matching method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more of the following image matching device embodiments can be found in the above-described limitations on the image matching method and will not be further elaborated here.

[0117] In one embodiment, Figure 5 As shown, an image matching device 500 is provided, comprising: a matching point acquisition module 510, a similarity acquisition module 520, a target pose acquisition module 530 and a matching module 540, wherein:

[0118] The matching point acquisition module 510 is used to obtain a projected two-dimensional matching point set and an original two-dimensional matching point set; wherein the projected two-dimensional matching point set is obtained by dimensionality reduction processing based on the three-dimensional matching point set corresponding to the three-dimensional feature image to be matched; the original two-dimensional matching point set is obtained based on the two-dimensional feature image to be matched.

[0119] The similarity acquisition module 520 is used to obtain the similarities between each projected two-dimensional matching point in the projected two-dimensional matching point set and the original two-dimensional matching point set, and to superimpose the similarities to obtain the target similarity.

[0120] The target posture acquisition module 530 is used to perform gradient descent processing on the target similarity, and obtain the target relative posture parameters when the target similarity meets the first preset condition.

[0121] The matching module 540 matches the three-dimensional feature image with the original two-dimensional feature image using the target relative posture parameters.

[0122] In one embodiment, the similarity acquisition module includes a matching point acquisition unit, a distance acquisition unit, and a similarity acquisition unit.

[0123] The matching point acquisition unit is configured to acquire each projected 2D matching point in the projected 2D matching point set and each original 2D matching point in the original 2D matching point set. The distance acquisition unit is configured to acquire distance parameters between each projected 2D matching point and each original 2D matching point. The similarity acquisition unit is configured to obtain the similarity between any projected 2D matching point and the original 2D matching point set based on the distance parameters between any projected 2D matching point and each original 2D matching point.

[0124] In one embodiment, the similarity acquisition unit includes a hyperparameter acquisition unit, a ratio unit, and a summation unit.

[0125] The hyperparameter acquisition unit is used to obtain preset hyperparameters. The ratio unit is used to obtain the similarity between any projected two-dimensional matching point and each original two-dimensional matching point based on the ratio of each distance parameter to the preset hyperparameter. The summation unit is used to obtain the similarity between any projected two-dimensional matching point and the set of original two-dimensional matching points based on the sum of the similarities of each matching point.

[0126] In one embodiment, the target pose acquisition module includes a Gaussian kernel function unit and a gradient descent processing unit.

[0127] The Gaussian kernel function unit is used to characterize the similarity of matching points using the Gaussian kernel function. The gradient descent processing unit is used to perform gradient descent processing on the target similarity using the Gaussian kernel function.

[0128] In one embodiment, the gradient descent processing unit is used to use the exponential function in the Gaussian kernel function as the numerator and the known two-norm as the denominator to obtain an iterative weight parameter; the known two-norm is obtained based on the projected two-dimensional matching point, the original two-dimensional matching point and the preset relative posture parameter; the unknown two-norm is obtained, and the iterative weight parameter is multiplied by the unknown two-norm to obtain a gradient descent function; according to the gradient descent function, a second-order gradient descent process is performed on the target similarity.

[0129] In one embodiment, the target posture acquisition module is used to obtain the target relative posture parameters when the gradient descent function meets the second preset condition.

[0130] In one embodiment, the gradient descent processing unit is configured to perform a second-order gradient descent process on the target similarity in a special Euclidean group according to the gradient descent function.

[0131] In one embodiment, the target pose acquisition module is used to obtain a target relative pose matrix in a special Euclidean group, and use the target relative pose matrix as a target relative pose parameter.

[0132] In one embodiment, the apparatus further comprises a coordinate conversion module.

[0133] The coordinate conversion module is used to convert the coordinates of the 3D matching point set to the coordinate system of the imaging device, thereby obtaining a set of 3D matching points after the coordinate conversion. The imaging device is used to obtain the 3D matching points. The dimensionality reduction processing module is used to perform dimensionality reduction processing on the 3D matching point set after the coordinate conversion using preset relative pose parameters, thereby obtaining a set of projected 2D matching points.

[0134] In one embodiment, the coordinate conversion module includes a conversion matrix acquisition unit and a conversion unit.

[0135] The transformation matrix acquisition unit is used to obtain the center of the initial coordinate system for the imaging device and the transformation matrix for the initial coordinate system. The conversion unit is used to transform each 3D matching point in the 3D matching point set using the center of the initial coordinate system and the transformation matrix to obtain a 3D matching point set after coordinate transformation.

[0136] Each module in the above-mentioned image matching device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0137] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store three-dimensional feature images and two-dimensional feature images. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image matching method is implemented.

[0138] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0139] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0140] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0141] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0142] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0143] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. An image matching method, characterized in that: The method comprises: Obtaining a projected two-dimensional matching point set and an original two-dimensional matching point set; wherein the projected two-dimensional matching point set is obtained by performing dimensionality reduction processing on a three-dimensional matching point set corresponding to a three-dimensional feature image to be matched; and the original two-dimensional matching point set is obtained based on the two-dimensional feature image to be matched; Obtaining each projected two-dimensional matching point in the projected two-dimensional matching point set and each original two-dimensional matching point in the original two-dimensional matching point set, obtaining distance parameters between each projected two-dimensional matching point and each original two-dimensional matching point, obtaining similarities between any projected two-dimensional matching point and the original two-dimensional matching point set based on the distance parameters between any projected two-dimensional matching point and each original two-dimensional matching point, obtaining similarities between each projected two-dimensional matching point in the projected two-dimensional matching point set and the original two-dimensional matching point set, and superimposing the similarities to obtain a target similarity; Performing a gradient descent process on the target similarity, and obtaining target relative pose parameters when the target similarity satisfies a first preset condition, further comprising: when the target similarity is maximized or the target similarity is within a preset range, using the relative pose parameters used for rotating and translating the three-dimensional feature image as the target relative pose parameters; The three-dimensional feature image and the original two-dimensional feature image are matched using the target relative posture parameters.

2. The method according to claim 1, characterized in that The obtaining, based on the distance parameters between any projected two-dimensional matching point and each of the original two-dimensional matching points, of the similarity between any projected two-dimensional matching point and the set of original two-dimensional matching points includes: Get the preset hyperparameters; Obtaining, according to the ratios of the respective distance parameters to the preset hyperparameters, respective similarities between the any projected two-dimensional matching point and the respective original two-dimensional matching points; The similarity between any projected two-dimensional matching point and the original two-dimensional matching point set is obtained according to the sum of the similarities of the respective matching points.

3. The method according to claim 2, characterized in that The performing gradient descent processing on the target similarity includes: Using a Gaussian kernel function to characterize the similarity of the matching points; The Gaussian kernel function is used to perform gradient descent processing on the target similarity.

4. The method according to claim 3, characterized in that The step of performing gradient descent processing on the target similarity by using the Gaussian kernel function includes: An exponential function in the Gaussian kernel function is used as a numerator and a known two-norm is used as a denominator to obtain an iterative weight parameter; the known two-norm is obtained based on the projected two-dimensional matching point, the original two-dimensional matching point and a preset relative pose parameter; Obtaining an unknown second norm, and multiplying the iterative weight parameter by the unknown second norm to obtain a gradient descent function; According to the gradient descent function, a second-order gradient descent process is performed on the target similarity.

5. The method according to claim 4, characterized in that The obtaining of target relative pose parameters includes: When the gradient descent function satisfies the second preset condition, the target relative pose parameters are obtained.

6. The method according to claim 4, characterized in that The step of performing a second-order gradient descent process on the target similarity according to the gradient descent function includes: According to the gradient descent function, a second-order gradient descent process is performed on the target similarity in a special Euclidean group.

7. The method according to claim 6, characterized in that The obtaining of target relative pose parameters includes: In the special Euclidean group, a target relative pose matrix is ​​obtained, and the target relative pose matrix is ​​used as the target relative pose parameter.

8. An image matching device, characterized in that: The device comprises: a matching point acquisition module, configured to acquire a projected two-dimensional matching point set and an original two-dimensional matching point set; wherein the projected two-dimensional matching point set is obtained by performing dimensionality reduction processing on the three-dimensional matching point set corresponding to the three-dimensional feature image to be matched; and the original two-dimensional matching point set is obtained based on the two-dimensional feature image to be matched; a similarity acquisition module, configured to acquire each projected two-dimensional matching point in the projected two-dimensional matching point set and each original two-dimensional matching point in the original two-dimensional matching point set, acquire distance parameters between each projected two-dimensional matching point and each original two-dimensional matching point, acquire similarities between any projected two-dimensional matching point and the original two-dimensional matching point set based on the distance parameters between the projected two-dimensional matching point and each original two-dimensional matching point, acquire similarities between each projected two-dimensional matching point in the projected two-dimensional matching point set and the original two-dimensional matching point set, and superimpose the similarities to acquire a target similarity; A target posture acquisition module is used to perform a gradient descent process on the target similarity, and obtain target relative posture parameters when the target similarity meets a first preset condition; The target pose acquisition module is further configured to, when the target similarity is maximized or the target similarity is within a preset range, use the relative pose parameters used to rotate and translate the three-dimensional feature image as the target relative pose parameters; A matching module matches the three-dimensional feature image with the original two-dimensional feature image through the target relative posture parameters.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images