Rigid registration parameter solving method and device, equipment and medium

Abstract

The invention provides a rigid registration parameter solving method and device, equipment and a medium. The rigid registration parameter solving method comprises the steps that a to-be-registered CT image and a target X-ray image are acquired; generating a plurality of digital reconstruction radiation images corresponding to the CT image to be registered based on a plurality of preset rigid registration partial solutions, and screening out a target digital reconstruction radiation image matched with the target X-ray image; acquiring a local digital reconstruction radiation image according to the target X-ray image, the target digital reconstruction radiation image and the corresponding rigid registration partial solution, and determining an initial rigid registration solution based on the local digital reconstruction radiation image and the target X-ray image; and performing adaptive fine tuning operation on the initial rigid registration solution to obtain a rigid registration solution set, and determining an optimal rigid registration solution in the rigid registration solution set until a preset condition is met. According to the method, more accurate rigid registration parameters can be provided, so that the image registration effect is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method, device, equipment and medium for solving rigid registration parameters. Background Art

[0002] Currently, in the field of medical image registration, traditional registration methods include registration methods using feature extraction, registration methods based on gray information, and registration methods based on deep learning. Among them, the registration method using feature extraction is difficult to achieve automation, the registration method based on gray information has poor registration effects on images with deformation or obvious gray changes, and the registration method based on deep learning is limited by the imaging machines of medical images, resulting in insufficient required data and also having problems with poor image registration effects. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, device, equipment and medium for solving rigid registration parameters, which can provide more accurate rigid registration parameters, thereby significantly improving the image registration effect.

[0004] In a first aspect, the present invention provides a method for solving rigid registration parameters, including:

[0005] Obtain the CT image to be registered and the target X-ray image;

[0006] Based on a plurality of preset partial rigid registration solutions, generate multiple digital reconstructed radiographs corresponding to the CT image to be registered, and screen out the target digital reconstructed radiograph that matches the target X-ray image; wherein, the partial rigid registration solution includes the rotation angle around the X-axis and the rotation angle around the Y-axis;

[0007] Obtain the local digital reconstructed radiograph according to the target X-ray image, the target digital reconstructed radiograph and its corresponding partial rigid registration solution, and determine the initial rigid registration solution based on the local digital reconstructed radiograph and the target X-ray image;

[0008] Perform an adaptive fine-tuning operation on the initial rigid registration solution to obtain a set of rigid registration solutions, and determine the optimal rigid registration solution in the set of rigid registration solutions until the preset condition is met; wherein, the preset condition is: the similarity between the new digital reconstructed radiograph generated based on the optimal rigid registration solution and the target X-ray image reaches the preset similarity threshold.

[0009] In an implementation manner, determining the initial rigid registration solution based on the local digital reconstructed radiograph and the target X-ray image includes:

[0010] For any three feature points in the target X-ray image, determine the feature points matching them in the local digital reconstructed radiograph to form feature point pairs, and determine the rotation angle about the Z-axis based on the vector pairs constructed from the feature point pairs;

[0011] Moreover, determine the scaling factor based on the feature points matching between the target X-ray image and the local digital reconstructed radiograph, and determine the translation amount along the Z-axis based on the scaling factor and the distance between the local digital reconstructed radiograph and the radiation source;

[0012] Moreover, determine the pixel movement distance coefficient corresponding to the movement per unit physical distance based on the template image with a known physical translation relationship, and determine the translation amounts along the X-axis and the Y-axis according to the pixel movement distance coefficient and the scaling factor;

[0013] Take the rotation angle about the X-axis, the rotation angle about the Y-axis, the rotation angle about the Z-axis, the translation amount along the X-axis, the translation amount along the Y-axis, and the translation amount along the Z-axis as the initial rigid registration solution.

[0014] In one implementation, for any three feature points in the target X-ray image, determine the feature points matching them in the local digital reconstructed radiograph to form feature point pairs, and determine the rotation angle about the Z-axis based on the vector pairs constructed from the feature point pairs, including:

[0015] Denote the three feature points in the target X-ray image as the first feature point, the second feature point, and the third feature point, and denote the feature points matching them in the local digital reconstructed radiograph as the fourth feature point, the fifth feature point, and the sixth feature point;

[0016] Construct a first vector pointing from the first feature point to the second feature point, a second vector pointing from the first feature point to the third feature point, a third vector pointing from the fourth feature point to the fifth feature point, and a fourth vector pointing from the fourth feature point to the sixth feature point;

[0017] Determine the angular deviation value between the included angle between the first vector and the third vector and the included angle between the second vector and the fourth vector;

[0018] Continue to extract new three feature points from the target X-ray image to obtain new angular deviation values, and take the average value of all angular deviation values as the rotation angle about the Z-axis.

[0019] In one implementation, determine the scaling factor based on the feature points matching between the target X-ray image and the local digital reconstructed radiograph, including:

[0020] Denote any feature point in the target X-ray image as the seventh feature point, and denote the feature point matching it in the local digital reconstructed radiograph as the eighth feature point;

[0021] Determine the connection line between the seventh feature point and the eighth feature point, the first length value in the target X-ray image, and the second length value in the local digital reconstructed radiograph, so as to obtain the length ratio between the first length value and the second length value;

[0022] Continue to extract new feature points from the target X-ray image to obtain new length values, and use the mean value of all length ratios as the scaling coefficient.

[0023] In one implementation, perform an adaptive fine-tuning operation on the initial rigid registration solution to obtain a set of rigid registration solutions, and determine the optimal rigid registration solution within the set of rigid registration solutions until the preset conditions are met, including:

[0024] Generate a new digital reconstructed radiograph corresponding to the CT image to be registered based on the initial rigid registration solution, and determine the similarity between the new digital reconstructed radiograph and the target X-ray;

[0025] If the similarity between the new digital reconstructed radiograph and the target X-ray does not meet the preset similarity threshold, determine the parameter fine-tuning range based on the similarity between the new digital reconstructed radiograph and the target X-ray;

[0026] Through each parameter fine-tuning model in the parameter fine-tuning model set, perform an adaptive fine-tuning operation on the initial rigid registration solution according to the parameter fine-tuning range to obtain a subset of rigid registration solutions output by each parameter fine-tuning model;

[0027] According to the fitness value of each rigid registration solution included in each subset of rigid registration solutions, screen multiple rigid registration solutions from each subset of rigid registration solutions to construct a set of rigid registration solutions;

[0028] Based on the distribution of the subsets of rigid registration solutions output by each parameter fine-tuning model in the set of rigid registration solutions, determine the allocation weight corresponding to each parameter fine-tuning model;

[0029] According to the allocation weight, allocate each rigid registration solution included in the set of rigid registration solutions to each parameter fine-tuning model, so as to continue to perform an adaptive fine-tuning operation on the rigid registration solution allocated to it by each parameter fine-tuning model according to the parameter fine-tuning range until the optimal rigid registration solution within the set of rigid registration solutions is determined when the preset conditions are met.

[0030] In one implementation, through each parameter fine-tuning model, continue to perform an adaptive fine-tuning operation on the rigid registration solution allocated to it according to the parameter fine-tuning range, including:

[0031] Based on the fine-tuning operators included in the fine-tuning parameter set and the adaptive fine-tuning control parameters used when performing an adaptive fine-tuning operation on the previous rigid registration solution, determine new adaptive fine-tuning control parameters;

[0032] Determine a new fine-tuning parameter according to the new adaptive fine-tuning control parameter;

[0033] Perform an adaptive fine-tuning operation on the current rigid registration solution by using the new fine-tuning parameter, and the rigid registration solution after the adaptive fine-tuning operation falls within the parameter fine-tuning range.

[0034] In one implementation, the method further includes:

[0035] When the fitness value of the new rigid registration solution obtained by the adaptive fine-tuning operation is higher than the fitness value of the current rigid registration solution, save the new fine-tuning parameter to the fine-tuning parameter set.

[0036] In a second aspect, the present invention further provides a rigid registration parameter solving device, including:

[0037] An image acquisition module, configured to acquire a CT image to be registered and a target X-ray image;

[0038] An image matching module, configured to generate multiple digital reconstructed radiographs corresponding to the CT image to be registered based on a plurality of preset partial rigid registration solutions, and screen out a target digital reconstructed radiograph that matches the target X-ray image; wherein, the partial rigid registration solution includes an angle of rotation around the X-axis and an angle of rotation around the Y-axis;

[0039] An initial solution determination module, configured to obtain a local digital reconstructed radiograph according to the target X-ray image, the target digital reconstructed radiograph and their corresponding partial rigid registration solutions, and determine an initial rigid registration solution based on the local digital reconstructed radiograph and the target X-ray image;

[0040] An adaptive fine-tuning module, configured to perform an adaptive fine-tuning operation on the initial rigid registration solution to obtain a set of rigid registration solutions, and determine the optimal rigid registration solution in the set of rigid registration solutions until a preset condition is met; wherein, the preset condition is: the similarity between the new digital reconstructed radiograph generated based on the optimal rigid registration solution and the target X-ray image reaches a preset similarity threshold.

[0041] In a third aspect, the present invention further provides an electronic device, including a processor and a memory, the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of the first aspect.

[0042] In a fourth aspect, the present invention further provides a computer-readable storage medium, the computer-readable storage medium stores computer executable instructions, and when the computer executable instructions are called and executed by the processor, the computer executable instructions cause the processor to implement the method according to any one of the first aspect.

[0043] A method, device, equipment and medium for solving rigid registration parameters provided by the present invention first obtain a CT image to be registered and a target X-ray image; then generate multiple digital reconstructed radiographs corresponding to the CT image to be registered based on a plurality of preset rigid registration partial solutions, and screen out the target digital reconstructed radiograph that matches the target X-ray image. The rigid registration partial solutions include the rotation angle around the X-axis and the rotation angle around the Y-axis; then obtain a local digital reconstructed radiograph according to the target X-ray image, the target digital reconstructed radiograph and its corresponding rigid registration partial solution, and determine an initial rigid registration solution based on the local digital reconstructed radiograph and the target X-ray image; finally, perform an adaptive fine-tuning operation on the initial rigid registration solution to obtain a set of rigid registration solutions, and determine the optimal rigid registration solution in the set of rigid registration solutions until a preset condition is met. The preset condition is that the similarity between the new digital reconstructed radiograph generated based on the optimal rigid registration solution and the target X-ray image reaches a preset similarity threshold. The above method generates multiple digital reconstructed radiographs corresponding to the CT image to be registered based on the rigid registration partial solutions, screens out the target digital reconstructed radiograph that matches the target X-ray image and its corresponding rigid registration partial solution through template matching, determines the rigid registration solution on this basis, and obtains a more accurate optimal rigid registration solution through iterative adaptive fine-tuning operations on the initial rigid registration solution, thereby significantly improving the image registration effect.

[0044] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by practicing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification, claims and drawings.

[0045] To make the above objectives, features and advantages of the present invention more obvious and understandable, the following specific preferred embodiments are given, and in conjunction with the accompanying drawings, the detailed description is as follows. Brief Description of the Drawings

[0046] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0047] Figure 1 It is a schematic flowchart of a method for solving rigid registration parameters provided by an embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram for solving the rotation angle around the Z-axis provided by an embodiment of the present invention;

[0049] Figure 3 Schematic diagram for solving the translation amount along the Z-axis provided by an embodiment of the present invention;

[0050] Figure 4 Schematic diagram for solving the translation amount along the X-axis and the translation amount along the Y-axis provided by an embodiment of the present invention;

[0051] Figure 5 Schematic structural diagram of a rigid registration parameter solving device provided by an embodiment of the present invention;

[0052] Figure 6 Schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0054] Currently, traditional registration methods have the problem of poor registration effects. Based on this, the embodiments of the present invention provide a rigid registration parameter solving method, device, equipment, and medium, which can provide more accurate rigid registration parameters, thereby significantly improving the image registration effect.

[0055] For ease of understanding of this embodiment, first, a rigid registration parameter solving method disclosed in the embodiments of the present invention will be introduced in detail. Refer to Figure 1 The flowchart of a rigid registration parameter solving method shown, and this method mainly includes the following steps S102 to step S108:

[0056] Step S102, obtain the CT image to be registered and the target X-ray image.

[0057] Step S104, based on a plurality of preset rigid registration partial decompositions, generate multiple digital reconstructed radiographs corresponding to the CT image to be registered, and screen out the target digital reconstructed radiograph that matches the target X-ray image.

[0058] Among them, the digitally reconstructed radiograph is abbreviated as DRR (Digitally Reconstructed Radiograph) image. The rigid registration partial solution includes the rotation angle around the X-axis and the rotation angle around the Y-axis. In one example, for each rigid registration partial solution, a digitally reconstructed radiograph based on the rotation of the X-axis and the Y-axis can be generated for the CT to be registered; then, feature point matching is performed between the target X-ray image and the digitally reconstructed radiograph; finally, based on the result of the feature point matching, template matching is performed between the target X-ray image and the digitally reconstructed radiograph to screen out the target digitally reconstructed radiograph that is most similar to the target X-ray image from the digitally reconstructed radiograph.

[0059] Step S106: Obtain a local digitally reconstructed radiograph according to the target X-ray image, the target digitally reconstructed radiograph, and their corresponding rigid registration partial solutions, and determine an initial rigid registration solution based on the local digitally reconstructed radiograph and the target X-ray image.

[0060] Among them, the local digitally reconstructed radiograph is obtained by cropping the target digitally reconstructed radiograph with the target X-ray image; the initial rigid registration solution includes the rotation angle around the X-axis, the rotation angle around the Y-axis, the rotation angle around the Z-axis, the translation amount along the X-axis, the translation amount along the Y-axis, and the translation amount along the Z-axis. In one example, after cropping the target digitally reconstructed radiograph with the target X-ray image, a local digitally reconstructed radiograph that matches the target X-ray image is obtained. Based on this, according to the geometric relationship between the target X-ray image and the local digitally reconstructed radiograph, and based on the rigid registration partial solution corresponding to the local digitally reconstructed radiograph, the rotation angle around the Z-axis, the translation amount along the X-axis, the translation amount along the Y-axis, and the translation amount along the Z-axis can be solved to obtain the initial rigid registration solution.

[0061] Step S108: Perform an adaptive fine-tuning operation on the initial rigid registration solution to obtain a set of rigid registration solutions, and determine the optimal rigid registration solution in the set of rigid registration solutions until the preset conditions are met.

[0062] Among them, the preset condition is that the similarity between the new digital reconstructed radiograph generated based on the optimal rigid registration solution and the target X-ray image reaches a preset similarity threshold. In one example, the initial rigid registration solution can be assigned to each parameter fine-tuning model in the parameter fine-tuning model set. Each parameter fine-tuning model adaptively fine-tunes the initial rigid registration solution with different algorithms to obtain a new rigid registration solution. The parameter fine-tuning model set can include one or more of a particle swarm model, a gradient descent model, a simulated annealing model, and a genetic model. Generate a new digital reconstructed radiograph corresponding to the CT image to be registered according to the new rigid registration solution, and use the similarity between the target X-ray image and the new digital reconstructed radiograph as the fitness value to evaluate the new rigid registration solution, so as to construct a new rigid registration solution set. Repeat the above process until the preset condition is met. At this time, the rigid registration solution with the highest fitness value in the rigid registration solution set can be used as the optimal rigid registration solution for registering the CT image to be registered, so that the registered CT image to be registered is aligned with the target X-ray image.

[0063] The rigid registration parameter solving method provided by the embodiment of the present invention generates multiple digital reconstructed radiographs corresponding to the CT image to be registered based on the partial rigid registration solution, and screens out the target digital reconstructed radiograph that matches the target X-ray image and its corresponding partial rigid registration solution through template matching. On this basis, the rigid registration solution is determined, and the initial rigid registration solution is iteratively adaptively fine-tuned to obtain a more accurate optimal rigid registration solution, thereby significantly improving the image registration effect.

[0064] For ease of understanding, the embodiment of the present invention provides a specific implementation manner of the rigid registration parameter solving method.

[0065] For the foregoing step S106, the following steps A1 to A4 can be executed to obtain a local digital reconstructed radiograph according to the target X-ray image, the target digital reconstructed radiograph, and its corresponding partial rigid registration solution, and determine the initial rigid registration solution based on the local digital reconstructed radiograph and the target X-ray image:

[0066] Step A1, for any three feature points in the target X-ray image, determine the feature points that match them in the local digital reconstructed radiograph to form a feature point pair, and determine the rotation angle around the Z axis based on the vector pair constructed by the feature point pair. The change controlled by the rotation angle around the Z axis is a rotation in the 2D relationship. Specifically, it includes the following steps A1.1 to A1.4:

[0067] Step A1.1, denote the three feature points in the target X-ray image as the first feature point, the second feature point, and the third feature point, and denote the feature points that match them in the local digital reconstructed radiograph as the fourth feature point, the fifth feature point, and the sixth feature point.

[0068] See Figure 2 the schematic diagram for solving the rotation angle about the Z-axis as shown, denote the three feature points in the target X-ray image as the first feature point O, the second feature point A, and the third feature point B, and denote the feature points determined to match them in the local digital reconstructed radiograph as the fourth feature point O', the fifth feature point A', and the sixth feature point B'.

[0069] Step A1.2, construct a first vector pointing from the first feature point to the second feature point, a second vector pointing from the first feature point to the third feature point, a third vector pointing from the fourth feature point to the fifth feature point, and a fourth vector pointing from the fourth feature point to the sixth feature point.

[0070] Please continue to see Figure 2 , after aligning the first feature point O and the fourth feature point O', construct the first vector OA, the second vector OB, the third vector O'A', and the fourth vector O'B'.

[0071] Step A1.3, determine the angular deviation value between the angle between the first vector and the third vector and the angle between the second vector and the fourth vector.

[0072] Please continue to see Figure 2 , determine the angle θ between the first vector OA and the third vector O'A', and determine the angle θ' between the second vector OB and the fourth vector O'B', and obtain the angular deviation value (i.e., the difference) between the angle θ and the angle θ'.

[0073] Step A1.4, continue to extract three new feature points from the target X-ray image to obtain a new angular deviation value, and take the average value of all angular deviation values as the rotation angle about the Z-axis. Similarly, calculate the angular deviation value between the corresponding two vectors combined by any two pairs of points, and the rotation angle r about the Z-axis can be obtained by calculating the average value of all vectors according to the following formula z :[[]]

[0074]

[0075] where r z is the rotation angle about the Z-axis, n is the total number of feature point pairs, and θ i is the angular deviation value of the i-th feature point pair.

[0076] Step A2, determine the scaling factor based on the feature points matched between the target X-ray image and the local digital reconstructed radiograph, and determine the translation amount along the Z-axis based on the scaling factor and the distance from the local digital reconstructed radiograph to the radiation source. See Figure 3Schematic diagram for solving the translation amount along the Z-axis. The reason why the imaging sizes of the target X-ray image and the local digital reconstructed radiograph are different is that the distances between the projection object and the radiation point are different during imaging. Dd is the distance from the local digital reconstructed radiograph to the radiation source, Dx is the distance from the target X-ray image to the radiation source during imaging, and Lxrary and Ldrr refer to the lengths of the connecting lines of the corresponding feature point pairs in the target X-ray image and the local digital reconstructed radiograph respectively. Specifically, it includes the following steps from Step A2.1 to Step A2.4:

[0077] Step A2.1, mark any feature point in the target X-ray image as the seventh feature point, and mark the feature point determined to match it in the local digital reconstructed radiograph as the eighth feature point. For example, mark any feature point in the target X-ray image as the seventh feature point C, and mark the feature point determined to match it in the local digital reconstructed radiograph as the eighth feature point C'.

[0078] Step A2.2, determine the connecting line C C' between the seventh feature point and the eighth feature point, and the first length value Lxrary in the target X-ray image and the second length value Ldrr in the local digital reconstructed radiograph respectively, to obtain the length ratio between the first length value and the second length value.

[0079] Step A2.3, continue to extract new feature points from the target X-ray image to obtain new length values, and take the average value of all length ratios as the scaling coefficient. Similarly, calculate the length values of the corresponding connecting lines combined by any two groups of point pairs, and the scaling coefficient r can be obtained by calculating the average value of the length values of all connecting lines according to the following formula:

[0080]

[0081] where r is the scaling coefficient, n is the total number of feature point pairs, L xrary is the length of the connecting line of the feature point pair in the target X-ray image, and L drr is the length of the connecting line of the feature point pair in the local digital reconstructed radiograph.

[0082] Step A2.4, determine the translation amount along the Z-axis based on the scaling coefficient and the distance from the local digital reconstructed radiograph to the radiation source. In one example, the translation amount along the Z-axis is determined according to the following formula:

[0083] t z =(r - 1)×D d ;

[0084] where t z is the translation amount along the Z-axis, r is the scaling coefficient, and D d is the distance from the local digital reconstructed radiograph to the radiation source.

[0085] Step A3: Based on the template image with known physical translation relationships, determine the pixel movement distance coefficient corresponding to each unit of physical distance movement. Then, determine the translation amounts along the X-axis and Y-axis according to the pixel movement distance coefficient and the scaling coefficient.

[0086] In one example, refer to Figure 4 the schematic diagram for solving the translation amounts along the X-axis and Y-axis as shown. Generate a new digital reconstructed radiograph based on the obtained translation and scaling parameters. It is necessary to calculate the image pixel movement distance coefficient p (abbreviated as physical movement ratio p) corresponding to each unit of physical distance movement according to the template image with known physical translation relationships. Then, when calculating the translation parameters, it is necessary to consider that the change in template scaling and cropping causes the physical movement ratio p to change accordingly. Therefore, use the scaling coefficient r for calculation, and the translation amount t x along the X-axis and the translation amount t y along the Y-axis in the new digital reconstructed radiograph can be obtained as follows:

[0087]

[0088] where p is the pixel movement distance coefficient, T x represents the physical movement distance of the template, t x ' represents the translation amount along the X-axis with known physical translation relationships, t x and t y are the translation amounts along the X-axis and Y-axis to be solved, x i and y i are the pixel coordinates in the local digital reconstructed radiograph, x i ′ and y i ′ are the pixel coordinates in the new digital reconstructed radiograph, and r is the scaling coefficient.

[0089] Step A4: Use the rotation angles around the X-axis, Y-axis, and Z-axis, and the translation amounts along the X-axis, Y-axis, and Z-axis as the initial rigid registration solution.

[0090] In one example, combine the rotation angle r x around the X-axis and the rotation angle r y around the Y-axis in the partial solution of rigid registration with the rotation angle r z around the Z-axis, the translation amount t x along the X-axis, the translation amount t y along the Y-axis, and the translation amount t z along the Z-axis obtained from the above Steps A1 to A3. Then, the initial rigid registration solution can be obtained.

[0091] For the foregoing step S108, the following steps B1 to B6 can be performed to adaptively fine-tune the initial rigid registration solution to obtain a set of rigid registration solutions, and determine the optimal rigid registration solution in the set of rigid registration solutions until the preset conditions are met:

[0092] Step B1: Generate a new digitally reconstructed radiograph corresponding to the CT image to be registered based on the initial rigid registration solution, and determine the similarity between the new digitally reconstructed radiograph and the target X-ray. The similarity between the new digitally reconstructed radiograph and the target X-ray can be evaluated using triangle similarity or structural similarity of the image.

[0093] Step B2: If the similarity between the new digitally reconstructed radiograph and the target X-ray does not meet the preset similarity threshold, determine the parameter fine-tuning range based on the similarity between the new digitally reconstructed radiograph and the target X-ray. Among them, the parameter fine-tuning range is positively correlated with the deviation between the similarity and the preset similarity threshold.

[0094] Step B3: Perform an adaptive fine-tuning operation on the initial rigid registration solution according to the parameter fine-tuning range through each parameter fine-tuning model in the parameter fine-tuning model set, and obtain a subset of rigid registration solutions output by each parameter fine-tuning model.

[0095] Exemplarily, the parameter fine-tuning model set includes a particle swarm model, a gradient descent model, a simulated annealing model, and a genetic model. The initial rigid registration solution is allocated to the particle swarm model, the gradient descent model, the simulated annealing model, and the genetic model, so as to perform at least one adaptive fine-tuning operation on the initial rigid registration solution through the above models using different algorithms, and obtain a subset of rigid registration solutions output by each parameter fine-tuning model.

[0096] Step B4: According to the fitness value of each rigid registration solution included in each subset of rigid registration solutions, screen multiple rigid registration solutions from each subset of rigid registration solutions to construct a set of rigid registration solutions.

[0097] In one example, a new digitally reconstructed radiograph can be generated based on the rigid registration solution, and the similarity between the new digitally reconstructed radiograph and the target X-ray image can be used as the fitness value of the rigid registration solution. Assume that only Q rigid registration solutions can be stored in the set of rigid registration solutions. Then, sort the rigid registration solutions included in all subsets of rigid registration solutions in descending order of fitness value, and include the first Q rigid registration solutions in the set of rigid registration solutions.

[0098] Step B5: Determine the allocation weight corresponding to each parameter fine-tuning model based on the distribution of the subset of rigid registration solutions output by each parameter fine-tuning model in the set of rigid registration solutions.

[0099] In one example, in the set of rigid registration solutions, the first quantity of the rigid registration solutions output by each model in the current iteration process can be counted, and the second quantity of the rigid registration solutions retained in the historical iteration process can be counted; the allocation weight, the first quantity, and the second quantity of each model in the current iteration process are input into a pre-trained weight prediction model, so that it outputs the allocation weight of each model in the next iteration process.

[0100] Further, before predicting the allocation weight of each model in the next iteration process, the sample data stored in the sample pool can be sampled, and the sampled samples can be used to dynamically update the weight prediction model. Exemplarily, the sample data includes the allocation weight of each model in the (k - 1)-th iteration process, the first quantity of the rigid registration solutions output by each model in the (k - 1)-th iteration process, the second quantity of the rigid registration solutions retained in the historical iteration process, and the allocation weight of each model in the k-th iteration process. A dynamically adjustable sampling probability is assigned to each sample data in the sample pool, and this sampling probability is related to its historical sampling situation. For example, if a sample data has not been sampled, the sampling probability corresponding to this sample data can be increased, and vice versa, the sampling probability corresponding to this sample data can be decreased, so as to sample sample data from the sample pool according to this sampling probability for dynamically updating the weight prediction model.

[0101] Step B6, according to the allocation weight, each rigid registration solution included in the set of rigid registration solutions is allocated to each parameter fine-tuning model, so that each parameter fine-tuning model continues to perform an adaptive fine-tuning operation on the allocated rigid registration solution according to the parameter fine-tuning range until the optimal rigid registration solution in the set of rigid registration solutions is determined when the preset condition is met.

[0102] In one example, the process of allocating each rigid registration solution included in the set of rigid registration solutions to each parameter fine-tuning model according to the allocation weight is as follows: based on the ratio of the allocation weight corresponding to the parameter fine-tuning model to the total allocation weight, a corresponding sector area is divided in a preset circular image, that is, the higher the allocation weight, the larger the area of the corresponding sector area; then, for any rigid registration solution in the set of rigid registration solutions, the divided circular image is used to randomly determine which parameter fine-tuning model the rigid registration solution is allocated to until all the rigid registration solutions in the set of rigid registration solutions are allocated.

[0103] In one example, the process of continuously performing adaptive fine-tuning operations on the assigned rigid registration solution according to the parameter fine-tuning range is as follows: Based on the fine-tuning operators included in the fine-tuning parameter set and the adaptive fine-tuning control parameters used when performing adaptive fine-tuning operations on the previous rigid registration solution, new adaptive fine-tuning control parameters are determined; new fine-tuning parameters are determined according to the new adaptive fine-tuning control parameters; the current rigid registration solution is adaptively fine-tuned using the new fine-tuning parameters, and the rigid registration solution after the adaptive fine-tuning operation falls within the parameter fine-tuning range.

[0104] Among them, the fine-tuning operator can be used as different model parameters in different parameter fine-tuning models. For example, in the particle swarm model, the fine-tuning operator can be used to control the degree to which particles approach the individual best position and the global best position; in the genetic model, the fine-tuning algorithm can be used to control the degree of crossover and mutation of individuals; in the simulated annealing model, the fine-tuning algorithm can be used to control the degree of random perturbation.

[0105] Exemplarily, F i representing the fine-tuning operator of the i-th original vector, will be re-initialized before the start of each iteration to complete the adaptive process, specifically as follows:

[0106] F i = rand c i (μF, 0.1);

[0107] where μF represents the adaptive fine-tuning control parameter, and rand c i (μ, σ 2 ) represents a value randomly selected from the normal distribution and the Cauchy distribution with mean μ and variance σ 2 .

[0108] If the benefit of the rigid registration solution after mutation is greater than the benefit of the original rigid registration solution, it proves that this is a successful fine-tuning operation. At this time, F i is stored in a set, represented by S F , and the update of μF in the next iteration is as follows:

[0109] μF = (1 - c) * μF + c * mean(S F );

[0110] where c is the weight and mean represents the arithmetic mean. At the start of each iteration, S F is cleared.

[0111] Furthermore, in the case where the fitness value of the new rigid registration solution obtained by the adaptive fine-tuning operation is higher than the fitness value of the current rigid registration solution, the new fine-tuning parameters are saved to the fine-tuning parameter set.

[0112] Repeat the above steps until the similarity between the digitally reconstructed radiograph generated based on the optimal rigid registration solution and the target X-ray image meets the preset similarity threshold, then the optimization can be stopped.

[0113] In summary, the embodiments of the present invention can obtain a more accurate optimal rigid registration solution, thereby significantly improving the image registration effect.

[0114] Based on the foregoing embodiments, an embodiment of the present invention provides a device for solving rigid registration parameters. Refer to Figure 5 the structural schematic diagram of a device for solving rigid registration parameters shown in

[0115] An image acquisition module 502, configured to acquire a CT image to be registered and a target X-ray image;

[0116] An image matching module 504, configured to generate multiple digitally reconstructed radiographs corresponding to the CT image to be registered based on a plurality of preset partial rigid registration solutions, and screen out a target digitally reconstructed radiograph that matches the target X-ray image; wherein, the partial rigid registration solutions include the rotation angle around the X-axis and the rotation angle around the Y-axis;

[0117] An initial solution determination module 506, configured to obtain a local digitally reconstructed radiograph according to the target X-ray image, the target digitally reconstructed radiograph and their corresponding partial rigid registration solutions, and determine an initial rigid registration solution based on the local digitally reconstructed radiograph and the target X-ray image;

[0118] An adaptive fine-tuning module 508, configured to perform an adaptive fine-tuning operation on the initial rigid registration solution to obtain a set of rigid registration solutions, and determine the optimal rigid registration solution in the set of rigid registration solutions until a preset condition is met; wherein, the preset condition is: the similarity between the newly generated digitally reconstructed radiograph based on the optimal rigid registration solution and the target X-ray image reaches the preset similarity threshold.

[0119] The device for solving rigid registration parameters provided by the embodiments of the present invention generates multiple digitally reconstructed radiographs corresponding to the CT image to be registered based on the partial rigid registration solutions, screens out the target digitally reconstructed radiograph that matches the target X-ray image and its corresponding partial rigid registration solutions through template matching, determines the rigid registration solution on this basis, and performs iterative adaptive fine-tuning operations on the initial rigid registration solution to obtain a more accurate optimal rigid registration solution, thereby significantly improving the image registration effect.

[0120] In an implementation manner, the initial solution determination module 506 is specifically configured to:

[0121] For any three feature points in the target X-ray image, determine the feature points matching them in the local digital reconstructed radiograph to form feature point pairs, and determine the rotation angle around the Z-axis based on the vector pairs constructed from the feature point pairs;

[0122] Moreover, determine the scaling factor based on the feature points matching between the target X-ray image and the local digital reconstructed radiograph, and determine the translation amount along the Z-axis based on the scaling factor and the distance between the local digital reconstructed radiograph and the radiation source;

[0123] Moreover, determine the pixel movement distance coefficient corresponding to the movement per unit physical distance based on the template image with a known physical translation relationship, and determine the translation amount along the X-axis and the translation amount along the Y-axis according to the pixel movement distance coefficient and the scaling factor;

[0124] Take the rotation angle around the X-axis, the rotation angle around the Y-axis, the rotation angle around the Z-axis, the translation amount along the X-axis, the translation amount along the Y-axis, and the translation amount along the Z-axis as the initial rigid registration solution.

[0125] In one implementation, the initial solution determination module 506 is specifically configured to:

[0126] Denote the three feature points in the target X-ray image as the first feature point, the second feature point, and the third feature point, and denote the feature points determined to match them in the local digital reconstructed radiograph as the fourth feature point, the fifth feature point, and the sixth feature point;

[0127] Construct a first vector pointing from the first feature point to the second feature point, a second vector pointing from the first feature point to the third feature point, a third vector pointing from the fourth feature point to the fifth feature point, and a fourth vector pointing from the fourth feature point to the sixth feature point;

[0128] Determine the angle deviation value between the angle between the first vector and the third vector and the angle between the second vector and the fourth vector;

[0129] Continue to extract new three feature points from the target X-ray image to obtain a new angle deviation value, and take the average value of all angle deviation values as the rotation angle around the Z-axis.

[0130] In one implementation, the initial solution determination module 506 is specifically configured to:

[0131] Denote any feature point in the target X-ray image as the seventh feature point, and denote the feature point determined to match it in the local digital reconstructed radiograph as the eighth feature point;

[0132] Determine the connection line between the seventh feature point and the eighth feature point, and respectively obtain the first length value in the target X-ray image and the second length value in the local digital reconstructed radiograph, so as to obtain the length ratio between the first length value and the second length value;

[0133] Continue to extract new feature points from the target X-ray image to obtain a new length ratio, and use the mean value of all length ratios as the scaling factor.

[0134] In one implementation, the adaptive fine-tuning module 508 is specifically configured to:

[0135] Generate a new digitally reconstructed radiograph corresponding to the CT image to be registered based on the initial rigid registration solution, and determine the similarity between the new digitally reconstructed radiograph and the target X-ray;

[0136] If the similarity between the new digitally reconstructed radiograph and the target X-ray does not meet the preset similarity threshold, determine the parameter fine-tuning range based on the similarity between the new digitally reconstructed radiograph and the target X-ray;

[0137] Perform an adaptive fine-tuning operation on the initial rigid registration solution according to the parameter fine-tuning range through each parameter fine-tuning model in the parameter fine-tuning model set to obtain a subset of rigid registration solutions output by each parameter fine-tuning model;

[0138] According to the fitness value of each rigid registration solution included in each subset of rigid registration solutions, screen multiple rigid registration solutions from each subset of rigid registration solutions to construct a set of rigid registration solutions;

[0139] Determine the allocation weight corresponding to each parameter fine-tuning model based on the distribution of the subset of rigid registration solutions output by each parameter fine-tuning model in the set of rigid registration solutions;

[0140] Allocate each rigid registration solution included in the set of rigid registration solutions to each parameter fine-tuning model according to the allocation weight, so that each parameter fine-tuning model continues to perform an adaptive fine-tuning operation on the rigid registration solution allocated to it according to the parameter fine-tuning range until the optimal rigid registration solution in the set of rigid registration solutions is determined when the preset condition is met.

[0141] In one implementation, the adaptive fine-tuning module 508 is specifically configured to:

[0142] Determine a new adaptive fine-tuning control parameter based on the fine-tuning operator included in the fine-tuning parameter set and the adaptive fine-tuning control parameter used when performing an adaptive fine-tuning operation on the previous rigid registration solution;

[0143] Determine a new fine-tuning parameter according to the new adaptive fine-tuning control parameter;

[0144] Perform an adaptive fine-tuning operation on the current rigid registration solution using the new fine-tuning parameter, and the rigid registration solution after the adaptive fine-tuning operation falls within the parameter fine-tuning range.

[0145] In one implementation, the adaptive fine-tuning module 508 is further configured to:

[0146] In the case where the fitness value of the new rigid registration solution obtained by the adaptive fine-tuning operation is higher than the fitness value of the current rigid registration solution, the new fine-tuning parameters are saved to the fine-tuning parameter set.

[0147] The device provided by the embodiments of the present invention has the same implementation principle and technical effects as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding contents in the foregoing method embodiments.

[0148] The embodiments of the present invention provide an electronic device. Specifically, the electronic device includes a processor and a storage device; a computer program is stored on the storage device, and the computer program, when run by the processor, executes the method according to any one of the foregoing embodiments.

[0149] Figure 6 FIG. 12 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. The electronic device 100 includes: a processor 60, a memory 61, a bus 62, and a communication interface 63. The processor 60, the communication interface 63, and the memory 61 are connected through the bus 62; the processor 60 is configured to execute an executable module stored in the memory 61, such as a computer program.

[0150] Among them, the memory 61 may include a high-speed random access memory (RAM, Random Access Memory), and may also include a non-volatile memory, such as at least one disk memory. Through at least one communication interface 63 (which may be wired or wireless), a communication connection between the system network element and at least one other network element is realized, and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used.

[0151] The bus 62 may be an ISA bus, a PCI bus, an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 6 only a bidirectional arrow is used in FIG. 12, but it does not mean that there is only one bus or one type of bus.

[0152] Among them, the memory 61 is used to store a program. After receiving an execution instruction, the processor 60 executes the program. The method executed by the device defined by the flow process disclosed in any one of the foregoing embodiments of the present invention can be applied to the processor 60 or implemented by the processor 60.

[0153] The processor 60 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 60 or the instructions in the form of software. The above-mentioned processor 60 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 61, and the processor 60 reads the information in the memory 61 and combines its hardware to complete the steps of the above method.

[0154] The computer program product of the readable storage medium provided by the embodiments of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the method described in the foregoing method embodiments. For the specific implementation, reference can be made to the foregoing method embodiments, and details are not described herein again.

[0155] If the above-described functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0156] Finally, it should be noted that the above-described embodiments are only specific implementation manners of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting them. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions described in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for solving rigid registration parameters, characterized in that: include: Acquire the CT image to be registered and the target X-ray image; Based on a plurality of preset rigid registration partial solutions, a plurality of digitally reconstructed radiographic images corresponding to the CT image to be registered are generated, and a target digitally reconstructed radiographic image matching the target X-ray image is screened out; wherein the rigid registration partial solutions include a rotation angle around an X-axis and a rotation angle around a Y-axis; Acquire a local digitally reconstructed radiographic image according to the target X-ray image, the target digitally reconstructed radiographic image and the corresponding rigid registration partial solution, and determine an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image; The initial rigid registration solution is adaptively fine-tuned to obtain a rigid registration solution set, and an optimal rigid registration solution within the rigid registration solution set is determined when a preset condition is met; wherein the preset condition is that the similarity between a new digitally reconstructed radiographic image generated based on the optimal rigid registration solution and the target X-ray image reaches a preset similarity threshold.

2. The method for solving rigid registration parameters according to claim 1, characterized in that: Determining an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image includes: For any three feature points in the target X-ray image, determining feature points matching the three feature points from the local digitally reconstructed radiographic image to form a feature point pair, and determining a rotation angle around the Z axis based on a vector pair constructed from the feature point pair; and determining a scaling factor based on feature points matched between the target X-ray image and the local digitally reconstructed radiographic image, and determining a translation amount along the Z axis based on the scaling factor and a distance between the local digitally reconstructed radiographic image and a radiation source; And, determining a pixel movement distance coefficient corresponding to each unit physical distance movement based on a template image with a known physical translation relationship, and determining a translation amount along the X axis and a translation amount along the Y axis according to the pixel movement distance coefficient and the scaling factor; The rotation angle around the X axis, the rotation angle around the Y axis, the rotation angle around the Z axis, the translation along the X axis, the translation along the Y axis and the translation along the Z axis are used as an initial rigid registration solution.

3. The method for solving rigid registration parameters according to claim 2, characterized in that: For any three feature points in the target X-ray image, feature points matching the three feature points are determined from the local digitally reconstructed radiographic image to form feature point pairs, and a rotation angle around the Z axis is determined based on a vector pair constructed from the feature point pairs, including: Recording the three feature points in the target X-ray image as a first feature point, a second feature point and a third feature point, and recording the feature points determined to match the target X-ray image in the local digitally reconstructed radiological image as a fourth feature point, a fifth feature point and a sixth feature point; Constructing a first vector pointing from the first feature point to the second feature point, a second vector pointing from the first feature point to the third feature point, a third vector pointing from the fourth feature point to the fifth feature point, and a fourth vector pointing from the fourth feature point to the sixth feature point; Determine an angle deviation value of an angle between the first vector and the third vector and an angle between the second vector and the fourth vector; Continue to extract three new feature points from the target X-ray image to obtain a new angle deviation value, and use the average of all the angle deviation values ​​as the rotation angle around the Z axis.

4. The method for solving rigid registration parameters according to claim 2, characterized in that: Determining a scaling factor based on feature points matched between the target X-ray image and the local digitally reconstructed radiographic image includes: Recording any feature point in the target X-ray image as the seventh feature point, and recording the feature point determined to match it in the local digitally reconstructed radiological image as the eighth feature point; Determine a first length value of a line between the seventh feature point and the eighth feature point in the target X-ray image and a second length value in the local digitally reconstructed radiographic image, respectively, to obtain a length ratio between the first length value and the second length value; New feature points are continuously extracted from the target X-ray image to obtain new length values, and the average of all the length ratios is used as the scaling factor.

5. The method for solving rigid registration parameters according to claim 1, characterized in that: Adaptively fine-tuning the initial rigid registration solution to obtain a rigid registration solution set, and determining the optimal rigid registration solution in the rigid registration solution set when a preset condition is met, including: Generating a new digitally reconstructed radiographic image corresponding to the CT image to be registered based on the initial rigid registration solution, and determining a similarity between the new digitally reconstructed radiographic image and the target X-ray; If the similarity between the new digitally reconstructed radiographic image and the target X-ray does not meet a preset similarity threshold, determining a parameter fine-tuning range based on the similarity between the new digitally reconstructed radiographic image and the target X-ray; By using each parameter fine-tuning model in the parameter fine-tuning model set, the initial rigid registration solution is adaptively fine-tuned according to the parameter fine-tuning range to obtain a rigid registration solution subset output by each parameter fine-tuning model; According to the fitness value of each rigid registration solution included in each rigid registration solution subset, a plurality of rigid registration solutions are selected from each rigid registration solution subset to construct a rigid registration solution set; Determine the allocation weight corresponding to each of the parameter fine-tuning models based on the distribution of the rigid registration solution subsets output by each of the parameter fine-tuning models in the rigid registration solution set; According to the assigned weight, each of the rigid registration solutions contained in the rigid registration solution set is assigned to each of the parameter fine-tuning models, so that through each of the parameter fine-tuning models, the rigid registration solutions assigned thereto continue to be adaptively fine-tuned according to the parameter fine-tuning range until the preset conditions are met, and the parameters at this time are determined to be the optimal rigid registration solution in the rigid registration solution set.

6. The method for solving rigid registration parameters according to claim 5, characterized in that: By means of each of the parameter fine-tuning models, the adaptive fine-tuning operation is continued on the rigid registration solution assigned thereto according to the parameter fine-tuning range, including: Determining new adaptive fine-tuning control parameters based on the fine-tuning operator included in the fine-tuning parameter set and the adaptive fine-tuning control parameters used when the adaptive fine-tuning operation is performed on the previous rigid registration solution; Determine a new fine-tuning parameter according to the new adaptive fine-tuning control parameter; The current rigid registration solution is adaptively fine-tuned using new fine-tuning parameters, so that the rigid registration solution after the adaptive fine-tuning operation falls within the parameter fine-tuning range.

7. The method for solving rigid registration parameters according to claim 6, characterized in that: The method further comprises: When the fitness value of the new rigid registration solution obtained by the adaptive fine-tuning operation is higher than the fitness value of the current rigid registration solution, the new fine-tuning parameters are saved in the fine-tuning parameter set.

8. A rigid registration parameter solving device, characterized in that: include: An image acquisition module, used to acquire the CT image to be registered and the target X-ray image; An image matching module, used for generating a plurality of digitally reconstructed radiographic images corresponding to the CT image to be registered based on a plurality of preset rigid registration partial solutions, and screening out a target digitally reconstructed radiographic image that matches the target X-ray image; wherein the rigid registration partial solution includes a rotation angle around an X-axis and a rotation angle around a Y-axis; An initial solution determination module, used for acquiring a local digitally reconstructed radiographic image according to the target X-ray image, the target digitally reconstructed radiographic image and the corresponding rigid registration partial solution, and determining an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image; An adaptive fine-tuning module is used to perform adaptive fine-tuning operations on the initial rigid registration solution to obtain a rigid registration solution set, and determine the optimal rigid registration solution in the rigid registration solution set when a preset condition is met; wherein the preset condition is: the similarity between the new digitally reconstructed radiographic image generated based on the optimal rigid registration solution and the target X-ray image reaches a preset similarity threshold.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 7.

Images