Rigid registration parameter solution method, device, equipment and medium

Abstract

The present invention provides a method, apparatus, device, and medium for solving rigid registration parameters, including: obtaining a CT image to be registered and a target X-ray image; generating multiple digitally reconstructed radiographic images corresponding to the CT image to be registered based on multiple preset rigid registration partial solutions, and screening out a target digitally reconstructed radiographic image that matches the target X-ray image; obtaining a local digitally reconstructed radiographic image based on the target X-ray image, the target digitally reconstructed radiographic image, and its corresponding rigid registration partial solution, and determining an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image; performing adaptive fine-tuning on the initial rigid registration solution to obtain a rigid registration solution set, and determining the optimal rigid registration solution within the rigid registration solution set when preset conditions are met. The present invention can provide more accurate rigid registration parameters, thereby significantly improving image registration effects.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and medium for solving rigid registration parameters. Background Art

[0002] At present, in the field of medical image registration, traditional registration methods include feature extraction registration methods, grayscale information-based registration methods, and deep learning-based registration methods. Among them, feature extraction registration methods are difficult to automate, grayscale information-based registration methods have poor registration effects on images with obvious deformation or grayscale changes, and deep learning-based registration methods are limited by the limitations of medical image imaging machines, resulting in insufficient required data and poor image registration effects. Summary of the Invention

[0003] In view of this, an object of the present invention is to provide a method, apparatus, device and medium for solving rigid registration parameters, which can provide more accurate rigid registration parameters, thereby significantly improving the image registration effect.

[0004] In a first aspect, the present invention provides a method for solving rigid registration parameters, comprising:

[0005] Obtain the CT image to be registered and the target X-ray image;

[0006] Generate multiple digitally reconstructed radiographic images corresponding to the CT image to be registered based on multiple preset rigid registration partial solutions, and select a target digitally reconstructed radiographic image that matches the target X-ray image; wherein the rigid registration partial solutions include rotation angles around the X-axis and rotation angles around the Y-axis;

[0007] obtaining a local digitally reconstructed radiographic image according to the target X-ray image, the target digitally reconstructed radiographic image, and their corresponding rigid registration partial solutions, and determining an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image;

[0008] An adaptive fine-tuning operation is performed on the initial rigid registration solution to obtain a rigid registration solution set, and the optimal rigid registration solution within the rigid registration solution set is determined when a preset condition is met; wherein the preset condition is that the similarity between the new digitally reconstructed radiographic image generated based on the optimal rigid registration solution and the target X-ray image reaches a preset similarity threshold.

[0009] In one embodiment, determining an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image includes:

[0010] For any three feature points in the target X-ray image, matching feature points are determined from the local digitally reconstructed radiographic image to form a feature point pair, and a rotation angle around the Z axis is determined based on a vector pair constructed from the feature point pair;

[0011] and determining a scaling factor based on matched feature points between the target X-ray image and the local digitally reconstructed radiographic image, and determining a translation amount along the Z axis based on the scaling factor and a distance between the local digitally reconstructed radiographic image and the radiation source;

[0012] and determining a pixel movement distance coefficient corresponding to each unit physical distance movement based on a template image with a known physical translation relationship, and determining a translation amount along the X axis and a translation amount along the Y axis according to the pixel movement distance coefficient and a scaling factor;

[0013] The rotation angle around the X axis, the rotation angle around the Y axis, the rotation angle around the Z axis, the translation along the X axis, the translation along the Y axis, and the translation along the Z axis are used as the initial rigid registration solution.

[0014] In one embodiment, for any three feature points in the target X-ray image, matching feature points are determined from the local digitally reconstructed radiographic image to form feature point pairs, and a rotation angle around the Z axis is determined based on a vector pair constructed from the feature point pairs, including:

[0015] The three feature points in the target X-ray image are recorded as the first feature point, the second feature point and the third feature point, and the feature points determined to match them in the local digitally reconstructed radiographic image are recorded as the fourth feature point, the fifth feature point and the sixth feature point;

[0016] Constructing a first vector pointing from the first feature point to the second feature point, a second vector pointing from the first feature point to the third feature point, a third vector pointing from the fourth feature point to the fifth feature point, and a fourth vector pointing from the fourth feature point to the sixth feature point;

[0017] Determine an angle deviation between the angle between the first vector and the third vector and the angle between the second vector and the fourth vector;

[0018] Continue to extract three new feature points from the target X-ray image to obtain a new angle deviation value, and take the average of all angle deviation values ​​as the rotation angle around the Z axis.

[0019] In one embodiment, determining a scaling factor based on matched feature points between a target X-ray image and a local digitally reconstructed radiographic image includes:

[0020] Any feature point in the target X-ray image is recorded as the seventh feature point, and the feature point determined to match it in the local digitally reconstructed radiographic image is recorded as the eighth feature point;

[0021] determining a first length value of a line connecting the seventh characteristic point and the eighth characteristic point in the target X-ray image and a second length value in the local digitally reconstructed radiographic image, respectively, to obtain a length ratio between the first length value and the second length value;

[0022] Continue to extract new feature points from the target X-ray image to obtain new length values, and use the average of all length ratios as the scaling factor.

[0023] In one embodiment, an adaptive fine-tuning operation is performed on the initial rigid registration solution to obtain a rigid registration solution set, and an optimal rigid registration solution in the rigid registration solution set is determined until a preset condition is satisfied, including:

[0024] generating a new digitally reconstructed radiographic image corresponding to the CT image to be registered based on the initial rigid registration solution, and determining a similarity between the new digitally reconstructed radiographic image and the target X-ray;

[0025] If the similarity between the new digitally reconstructed radiographic image and the target X-ray does not meet a preset similarity threshold, determining a parameter fine-tuning range based on the similarity between the new digitally reconstructed radiographic image and the target X-ray;

[0026] Adaptively fine-tune the initial rigid registration solution according to the parameter fine-tuning range through each parameter fine-tuning model in the parameter fine-tuning model set, and obtain a subset of rigid registration solutions output by each parameter fine-tuning model;

[0027] According to the fitness value of each rigid registration solution contained in each rigid registration solution subset, a plurality of rigid registration solutions are screened from each rigid registration solution subset to construct a rigid registration solution set;

[0028] Determine the allocation weight corresponding to each parameter fine-tuning model based on the distribution of the rigid registration solution subset output by each parameter fine-tuning model in the rigid registration solution set;

[0029] According to the assigned weight, each rigid registration solution contained in the rigid registration solution set is assigned to each parameter fine-tuning model, so that each parameter fine-tuning model continues to perform adaptive fine-tuning operations on the assigned rigid registration solution according to the parameter fine-tuning range until the optimal rigid registration solution in the rigid registration solution set is determined when the preset conditions are met.

[0030] In one embodiment, each parameter fine-tuning model continues to adaptively fine-tune its assigned rigid registration solution according to the parameter fine-tuning range, including:

[0031] Determining new adaptive fine-tuning control parameters based on the fine-tuning operators included in the fine-tuning parameter set and the adaptive fine-tuning control parameters used when performing the adaptive fine-tuning operation on the previous rigid registration solution;

[0032] determining new fine-tuning parameters according to the new adaptive fine-tuning control parameters;

[0033] The new fine-tuning parameters are used to perform an adaptive fine-tuning operation on the current rigid registration solution, and the rigid registration solution after the adaptive fine-tuning operation falls within the parameter fine-tuning range.

[0034] In one embodiment, the method further comprises:

[0035] When the fitness value of the new rigid registration solution obtained by the adaptive fine-tuning operation is higher than the fitness value of the current rigid registration solution, the new fine-tuning parameters are saved in the fine-tuning parameter set.

[0036] In a second aspect, the present invention further provides a rigid registration parameter solving device, comprising:

[0037] An image acquisition module is used to acquire the CT image to be registered and the target X-ray image;

[0038] An image matching module is configured to generate multiple digitally reconstructed radiographic images corresponding to the CT image to be registered based on multiple preset rigid registration partial solutions, and to select a target digitally reconstructed radiographic image that matches the target X-ray image; wherein the rigid registration partial solutions include rotation angles around the X-axis and rotation angles around the Y-axis;

[0039] an initial solution determination module, configured to obtain a local digitally reconstructed radiographic image based on a target X-ray image, a target digitally reconstructed radiographic image, and their corresponding rigid registration partial solutions, and determine an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image;

[0040] The adaptive fine-tuning module is used to perform adaptive fine-tuning operations on the initial rigid registration solution to obtain a rigid registration solution set, and determine the optimal rigid registration solution within the rigid registration solution set when a preset condition is met; wherein the preset condition is: the similarity between the new digitally reconstructed radiographic image generated based on the optimal rigid registration solution and the target X-ray image reaches a preset similarity threshold.

[0041] In a third aspect, the present invention further provides an electronic device comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement any one of the methods provided in the first aspect.

[0042] In a fourth aspect, the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods provided in the first aspect.

[0043] The present invention provides a method, device, equipment and medium for solving rigid registration parameters. First, a CT image to be registered and a target X-ray image are obtained; then, based on multiple preset rigid registration partial solutions, multiple digitally reconstructed radiographic images corresponding to the CT image to be registered are generated, and a target digitally reconstructed radiographic image that matches the target X-ray image is screened out, wherein the rigid registration partial solution includes a rotation angle around the X-axis and a rotation angle around the Y-axis; then, a local digitally reconstructed radiographic image is obtained based on the target X-ray image, the target digitally reconstructed radiographic image and its corresponding rigid registration partial solution, and an initial rigid registration solution is determined based on the local digitally reconstructed radiographic image and the target X-ray image; finally, an adaptive fine-tuning operation is performed on the initial rigid registration solution to obtain a rigid registration solution set, and an optimal rigid registration solution within the rigid registration solution set is determined when a preset condition is met. The preset condition is: the similarity between the new digitally reconstructed radiographic image generated based on the optimal rigid registration solution and the target X-ray image reaches a preset similarity threshold. The above method generates multiple digitally reconstructed radiographic images corresponding to the CT image to be registered based on the rigid registration partial solution, and screens out the target digitally reconstructed radiographic images and their corresponding rigid registration partial solutions that match the target X-ray image through template matching. On this basis, the rigid registration solution is determined, and the initial rigid registration solution is iteratively adaptively fine-tuned to obtain a more accurate optimal rigid registration solution, thereby significantly improving the image registration effect.

[0044] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or understood by practicing the present invention. The purposes and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A schematic flow chart of a method for solving rigid registration parameters provided by an embodiment of the present invention;

[0048] Figure 2 A schematic diagram of a method for calculating a rotation angle around the Z axis provided by an embodiment of the present invention;

[0049] Figure 3 A schematic diagram of solving the translation amount along the Z axis provided by an embodiment of the present invention;

[0050] Figure 4 A schematic diagram of solving the translation amount along the X-axis and the translation amount along the Y-axis provided by an embodiment of the present invention;

[0051] Figure 5 A schematic structural diagram of a rigid registration parameter solving device provided by an embodiment of the present invention;

[0052] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0054] At present, traditional registration methods have the problem of poor registration effect. Based on this, the present invention implements a method, device, equipment and medium for solving rigid registration parameters, which can provide more accurate rigid registration parameters, thereby significantly improving the image registration effect.

[0055] To facilitate understanding of this embodiment, a method for solving rigid registration parameters disclosed in an embodiment of the present invention is first described in detail. Figure 1 The flowchart of a method for solving rigid registration parameters is shown in FIG. 1 , which mainly includes the following steps S102 to S108:

[0056] Step S102: Acquire the CT image to be registered and the target X-ray image.

[0057] Step S104 : generating a plurality of digitally reconstructed radiographic images corresponding to the CT image to be registered based on the preset plurality of rigid registration partial solutions, and screening out a target digitally reconstructed radiographic image that matches the target X-ray image.

[0058] The digitally reconstructed radiograph (DRR) image is abbreviated as a DRR image, and the rigid registration partial solution includes the rotation angle around the X-axis and the rotation angle around the Y-axis. In one example, for each rigid registration partial solution, a digitally reconstructed radiograph image based on the X-axis and Y-axis rotations can be generated for the target CT. Feature point matching is then performed between the target X-ray image and the digitally reconstructed radiograph image. Finally, based on the feature point matching results, template matching is performed between the target X-ray image and the digitally reconstructed radiograph image to select the target digitally reconstructed radiograph image that is most similar to the target X-ray image from the digitally reconstructed radiograph images.

[0059] Step S106 , obtaining a local digitally reconstructed radiographic image according to the target X-ray image, the target digitally reconstructed radiographic image and their corresponding rigid registration partial solutions, and determining an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image.

[0060] The local digitally reconstructed radiographic image is obtained by cropping the target digitally reconstructed radiographic image using the target X-ray image; the initial rigid registration solution includes a rotation angle about the X-axis, a rotation angle about the Y-axis, a rotation angle about the Z-axis, a translation along the X-axis, a translation along the Y-axis, and a translation along the Z-axis. In one example, after cropping the target digitally reconstructed radiographic image based on the target X-ray image, a local digitally reconstructed radiographic image matching the target X-ray image is obtained. Based on this, the rotation angle about the Z-axis, the translation along the X-axis, the translation along the Y-axis, and the translation along the Z-axis can be solved based on the geometric relationship between the target X-ray image and the local digitally reconstructed radiographic image and the rigid registration partial solution corresponding to the local digitally reconstructed radiographic image to obtain an initial rigid registration solution.

[0061] Step S108 : Adaptively fine-tuning the initial rigid registration solution to obtain a rigid registration solution set, and determining the optimal rigid registration solution in the rigid registration solution set when a preset condition is met.

[0062] The preset condition is that the similarity between the new digitally reconstructed radiographic image generated based on the optimal rigid registration solution and the target X-ray image reaches a preset similarity threshold. In one example, the initial rigid registration solution can be assigned to each parameter fine-tuning model in a parameter fine-tuning model set. Each parameter fine-tuning model adaptively fine-tunes the initial rigid registration solution using a different algorithm to obtain a new rigid registration solution. The parameter fine-tuning model set can include one or more of a particle swarm model, a gradient descent model, a simulated annealing model, and a genetic model. A new digitally reconstructed radiographic image corresponding to the CT image to be registered is generated according to the new rigid registration solution. The similarity between the target X-ray image and the new digitally reconstructed radiographic image is used as a fitness value to evaluate the new rigid registration solution to construct a new rigid registration solution set. The above process is repeated until the preset condition is met. The rigid registration solution with the highest fitness value in the rigid registration solution set is then used as the optimal rigid registration solution for registering the CT image to be registered, so that the registered CT image to be registered is aligned with the target X-ray image.

[0063] The rigid registration parameter solution method provided in the embodiment of the present invention generates multiple digitally reconstructed radiographic images corresponding to the CT image to be registered based on the rigid registration partial solution, screens out the target digitally reconstructed radiographic image and its corresponding rigid registration partial solution that matches the target X-ray image through template matching, determines the rigid registration solution on this basis, and obtains a more accurate optimal rigid registration solution by iteratively adaptively fine-tuning the initial rigid registration solution, thereby significantly improving the image registration effect.

[0064] For ease of understanding, an embodiment of the present invention provides a specific implementation of a method for solving rigid registration parameters.

[0065] For the aforementioned step S106, the following steps A1 to A4 may be performed to obtain a local digitally reconstructed radiographic image based on the target X-ray image, the target digitally reconstructed radiographic image, and their corresponding rigid registration partial solutions, and determine an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image:

[0066] Step A1: For any three feature points in the target X-ray image, matching feature points are determined from the local digitally reconstructed radiographic image to form feature point pairs. The rotation angle around the Z axis is determined based on the vector pairs constructed from the feature point pairs. The change in the rotation angle around the Z axis is a 2D rotation. This specifically includes the following steps A1.1 to A1.4:

[0067] In step A1.1, the three feature points in the target X-ray image are recorded as the first feature point, the second feature point, and the third feature point, and the feature points determined to match them in the local digitally reconstructed radiographic image are recorded as the fourth feature point, the fifth feature point, and the sixth feature point.

[0068] See also Figure 2 A schematic diagram of solving the rotation angle around the Z axis is shown, in which the three feature points in the target X-ray image are recorded as the first feature point O, the second feature point A, and the third feature point B, and the feature points determined to match them in the local digitally reconstructed radiographic image are recorded as the fourth feature point O', the fifth feature point A', and the sixth feature point B'.

[0069] Step A1.2: construct a first vector pointing from the first feature point to the second feature point, a second vector pointing from the first feature point to the third feature point, a third vector pointing from the fourth feature point to the fifth feature point, and a fourth vector pointing from the fourth feature point to the sixth feature point.

[0070] Please continue to see Figure 2 After aligning the first feature point O and the fourth feature point O', a first vector OA, a second vector OB, a third vector O'A', and a fourth vector O'B' are constructed.

[0071] Step A1.3: Determine the angle deviation between the angle between the first vector and the third vector and the angle between the second vector and the fourth vector.

[0072] Please continue to see Figure 2 , determine the angle θ between the first vector OA and the third vector O'A', and determine the angle θ' between the second vector OB and the fourth vector O'B', and obtain the angle deviation value (ie, the difference) between the angle θ and the angle θ'.

[0073] In step A1.4, continue to extract three new feature points from the target X-ray image to obtain new angle deviation values. The average of all angle deviation values ​​is used as the rotation angle around the Z axis. Similarly, calculate the angle deviation value of the corresponding two vectors from any two sets of point pairs. The rotation angle r around the Z axis can be obtained by calculating the average value of all vectors according to the following formula: z :

[0074]

[0075] Among them, r z is the rotation angle around the Z axis, n is the total number of feature point pairs, θ i is the angle deviation value of the i-th feature point pair.

[0076] Step A2: Determine the scaling factor based on the matching feature points between the target X-ray image and the local digitally reconstructed radiographic image, and determine the translation along the Z axis based on the scaling factor and the distance between the local digitally reconstructed radiographic image and the radiation source. Figure 3The diagram below shows a schematic diagram for solving the Z-axis translation. The difference in size between the target X-ray image and the local digitally reconstructed radiographic image is due to the different distances between the projected object and the radiation point during imaging. Dd is the distance from the local digitally reconstructed radiographic image to the radiation source, Dx is the distance from the target X-ray image to the radiation source during imaging, and Lxrary and Ldrr are the lengths of the lines connecting the corresponding feature point pairs in the target X-ray image and the local digitally reconstructed radiographic image, respectively. This includes steps A2.1 through A2.4:

[0077] In step A2.1, any feature point in the target X-ray image is recorded as the seventh feature point, and the feature point determined to match it in the local digitally reconstructed radiographic image is recorded as the eighth feature point. For example, any feature point in the target X-ray image is recorded as the seventh feature point C, and the feature point determined to match it in the local digitally reconstructed radiographic image is recorded as the eighth feature point C'.

[0078] In step A2.2, a line C C' between the seventh feature point and the eighth feature point is determined to have a first length value Lxrary in the target X-ray image and a second length value Ldrr in the local digitally reconstructed radiographic image, respectively, to obtain a length ratio between the first length value and the second length value.

[0079] In step A2.3, continue extracting new feature points from the target X-ray image to obtain new length values, and use the average of all length ratios as the scaling factor. Similarly, calculate the length of the corresponding line formed by any two sets of point pairs. The scaling factor r can be obtained by calculating the average of the length values ​​of all lines according to the following formula:

[0080]

[0081] Among them, r is the scaling factor, n is the total number of feature point pairs, L xrary is the length of the line connecting the feature point pairs in the target X-ray image, L drr is the length of the line connecting the feature point pairs in the local digitally reconstructed radiographic image.

[0082] In step A2.4, the Z-axis translation is determined based on the zoom factor and the distance between the local digitally reconstructed radiographic image and the radiation source. In one example, the Z-axis translation is determined according to the following formula:

[0083] t z =(r-1)×D d ;

[0084] Among them, t z is the translation along the Z axis, r is the scaling factor, D d The distance from the local digitally reconstructed radiographic image to the radiation source.

[0085] Step A3: Determine the pixel movement distance coefficient corresponding to each unit physical distance movement based on the template image with known physical translation relationship, and determine the translation amount along the X axis and the translation amount along the Y axis according to the pixel movement distance coefficient and the scaling factor.

[0086] In one example, see Figure 4 The schematic diagram of solving the translation along the X axis and the translation along the Y axis is shown. A new digitally reconstructed radiographic image is generated based on the obtained translation and scaling parameters. It is necessary to calculate the image pixel movement distance coefficient p corresponding to each unit physical distance movement based on the template image with a known physical translation relationship, which is referred to as the physical movement ratio p. Then, when calculating the translation parameters, it is also necessary to take into account the change in the template scaling and cropping, which causes the physical movement ratio p to change accordingly. Therefore, the scaling coefficient r is used for calculation to obtain the translation along the X axis t in the new digitally reconstructed radiographic image. x and the translation along the Y axis t y :

[0087]

[0088] Among them, p is the pixel movement distance coefficient, T x Indicates the distance the template physically moves, t x ′ represents the translation along the X axis of the known physical review relationship, t x , t y The translation along the X axis and the Y axis to be solved, x i 、y i is the pixel coordinate in the local digitally reconstructed radiographic image, x i ′ 、y i ′ are the pixel coordinates in the new digitally reconstructed radiographic image, and r is the scaling factor.

[0089] In step A4, the rotation angle around the X axis, the rotation angle around the Y axis, the rotation angle around the Z axis, the translation along the X axis, the translation along the Y axis, and the translation along the Z axis are used as the initial rigid registration solution.

[0090] In one example, the rotation angle r around the X axis in the rigid registration solution is x , rotation angle r around the Y axis y , and the rotation angle r around the Z axis obtained from steps A1 to A3 above z , translation along the X axis t x , translation along the Y axis t y and the translation along the Z axis t z By combining them, we can get the initial rigid registration solution.

[0091] For the aforementioned step S108, the initial rigid registration solution may be adaptively fine-tuned according to the following steps B1 to B6 to obtain a rigid registration solution set, and the optimal rigid registration solution in the rigid registration solution set may be determined when a preset condition is met:

[0092] In step B1, a new digitally reconstructed radiographic image corresponding to the CT image to be registered is generated based on the initial rigid registration solution, and the similarity between the new digitally reconstructed radiographic image and the target X-ray is determined. The similarity between the new digitally reconstructed radiographic image and the target X-ray can be evaluated using triangle similarity and image structure similarity.

[0093] In step B2, if the similarity between the new digitally reconstructed radiographic image and the target X-ray does not meet the preset similarity threshold, a parameter fine-tuning range is determined based on the similarity between the new digitally reconstructed radiographic image and the target X-ray, wherein the parameter fine-tuning range is positively correlated with the deviation between the similarity and the preset similarity threshold.

[0094] In step B3, adaptive fine-tuning is performed on the initial rigid registration solution according to the parameter fine-tuning range through each parameter fine-tuning model in the parameter fine-tuning model set to obtain a subset of rigid registration solutions output by each parameter fine-tuning model.

[0095] Exemplarily, the parameter fine-tuning model set includes a particle swarm model, a gradient descent model, a simulated annealing model, and a genetic model. The initial rigid registration solution is distributed to the particle swarm model, the gradient descent model, the simulated annealing model, and the genetic model, so that the initial rigid registration solution is adaptively fine-tuned at least once using different algorithms through the above models to obtain a subset of rigid registration solutions output by the parameter fine-tuning model.

[0096] Step B4: screening multiple rigid registration solutions from each rigid registration solution subset according to the fitness value of each rigid registration solution contained in each rigid registration solution subset to construct a rigid registration solution set.

[0097] In one example, a new digitally reconstructed radiographic image can be generated based on the rigid registration solution, and the similarity between the new digitally reconstructed radiographic image and the target X-ray image is used as the fitness value of the rigid registration solution. Assuming that only Q rigid registration solutions can be stored in the rigid registration solution set, the rigid registration solutions contained in all rigid registration solution subsets are sorted in descending order of fitness value, and the top Q rigid registration solutions are included in the rigid registration solution set.

[0098] Step B5: determining the allocation weight corresponding to each parameter fine-tuning model based on the distribution of the rigid registration solution subset output by each parameter fine-tuning model in the rigid registration solution set.

[0099] In one example, the first number of rigid registration solutions output by each model in the current iteration process and the second number of rigid registration solutions retained in the historical iteration process in the rigid registration solution set can be counted; the allocation weight, the first number, and the second number of each model in the current iteration process are input into the pre-trained weight prediction model to enable it to output the allocation weight of each model in the next iteration process.

[0100] Furthermore, before predicting the allocated weight of each model in the next iteration, the sample data stored in the sample pool can be sampled, and the weight prediction model can be dynamically updated using the sampled samples. Exemplarily, the sample data includes the allocated weight of each model in the k-1th iteration, the first number of rigid registration solutions output by each model in the k-1th iteration, the second number of rigid registration solutions retained from the historical iteration process, and the allocated weight of each model in the k-th iteration. A dynamically adjustable sampling probability is assigned to each sample data in the sample pool. The sampling probability is related to its historical sampling situation. For example, if a certain sample data has not been sampled, the sampling probability corresponding to the sample data can be increased, and vice versa, the sampling probability corresponding to the sample data can be reduced, so as to sample sample data from the sample pool according to the sampling probability for dynamic updating of the weight prediction model.

[0101] In step B6, each rigid registration solution contained in the rigid registration solution set is assigned to each parameter fine-tuning model according to the assigned weight, so that each parameter fine-tuning model continues to perform adaptive fine-tuning operations on the rigid registration solution assigned to it according to the parameter fine-tuning range until the optimal rigid registration solution in the rigid registration solution set is determined when the preset conditions are met.

[0102] In one example, the process of assigning each rigid registration solution contained in the rigid registration solution set to each parameter fine-tuning model according to the assigned weight is as follows: based on the ratio of the assigned weight corresponding to the parameter fine-tuning model to the total assigned weight, a corresponding sector-shaped area is divided in the preset circular image, that is, the higher the assigned weight, the larger the area of ​​the corresponding sector-shaped area; then, for any rigid registration solution in the rigid registration solution set, the divided circular image is used to randomly determine to which parameter fine-tuning model the rigid registration solution is assigned, until all rigid registration solutions in the rigid registration solution set are assigned.

[0103] In one example, the process of continuing to perform adaptive fine-tuning operations on the rigid registration solution assigned to it according to the parameter fine-tuning range is as follows: based on the fine-tuning operators contained in the fine-tuning parameter set and the adaptive fine-tuning control parameters used when the previous rigid registration solution was adaptively fine-tuned, new adaptive fine-tuning control parameters are determined; new fine-tuning parameters are determined according to the new adaptive fine-tuning control parameters; and the new fine-tuning parameters are used to perform adaptive fine-tuning operations on the current rigid registration solution, so that the rigid registration solution after the adaptive fine-tuning operation falls within the parameter fine-tuning range.

[0104] Fine-tuning operators can be used as different model parameters in different parameter fine-tuning models. For example, in particle swarm models, fine-tuning operators can be used to control the degree to which particles approach their individual optimal positions and the global optimal position; in genetic models, fine-tuning algorithms can be used to control the degree of crossover and mutation of individuals; and in simulated annealing models, fine-tuning algorithms can be used to control the degree of random perturbation.

[0105] For example, F i The fine-tuning operator representing the i-th original vector will be reinitialized before each iteration to complete the adaptive process, as follows:

[0106] F i =rand c i (μF,0.1);

[0107] Where μF represents the adaptive fine-tuning control parameter, rand c i (μ,σ 2 ) represents the mean μ and variance σ 2 Randomly selected values ​​from the normal and Cauchy distributions.

[0108] If the benefit of the mutated rigid registration solution is greater than the benefit of the original rigid registration solution, it proves that this is a successful fine-tuning operation, and F i Stored in a collection, using S F Indicates that the update of μF in the next iteration is as follows:

[0109] μF=(1-c)*μF+c*mean(S F );

[0110] Among them, c is the weight, mean represents the arithmetic mean, and at the beginning of each iteration, S is cleared. F .

[0111] Furthermore, when the fitness value of the new rigid registration solution obtained by the adaptive fine-tuning operation is higher than the fitness value of the current rigid registration solution, the new fine-tuning parameters are saved in the fine-tuning parameter set.

[0112] Repeat the above steps until the similarity between the digitally reconstructed radiographic image generated based on the optimal rigid registration solution and the target X-ray image meets the preset similarity threshold, and then stop the optimization.

[0113] In summary, the embodiments of the present invention can obtain a more accurate optimal rigid registration solution, thereby significantly improving the image registration effect.

[0114] Based on the above embodiment, the present invention provides a rigid registration parameter solving device. Figure 5 The schematic diagram of the structure of a rigid registration parameter solving device shown in FIG. 1 mainly includes the following parts:

[0115] An image acquisition module 502 is used to acquire the CT image to be registered and the target X-ray image;

[0116] An image matching module 504 is configured to generate multiple digitally reconstructed radiographic images corresponding to the CT image to be registered based on multiple preset rigid registration partial solutions, and to select a target digitally reconstructed radiographic image that matches the target X-ray image; wherein the rigid registration partial solutions include rotation angles around the X-axis and rotation angles around the Y-axis;

[0117] an initial solution determination module 506 for obtaining a local digitally reconstructed radiographic image based on the target X-ray image, the target digitally reconstructed radiographic image, and their corresponding rigid registration partial solutions, and determining an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image;

[0118] The adaptive fine-tuning module 508 is used to perform adaptive fine-tuning operations on the initial rigid registration solution to obtain a rigid registration solution set, and determine the optimal rigid registration solution in the rigid registration solution set when a preset condition is met; wherein the preset condition is: the similarity between the new digitally reconstructed radiographic image generated based on the optimal rigid registration solution and the target X-ray image reaches a preset similarity threshold.

[0119] The rigid registration parameter solving device provided in an embodiment of the present invention generates multiple digitally reconstructed radiographic images corresponding to the CT image to be registered based on the rigid registration partial solution, screens out the target digitally reconstructed radiographic image and its corresponding rigid registration partial solution that matches the target X-ray image through template matching, determines the rigid registration solution on this basis, and obtains a more accurate optimal rigid registration solution by iteratively adaptively fine-tuning the initial rigid registration solution, thereby significantly improving the image registration effect.

[0120] In one embodiment, the initial solution determination module 506 is specifically configured to:

[0121] For any three feature points in the target X-ray image, matching feature points are determined from the local digitally reconstructed radiographic image to form a feature point pair, and a rotation angle around the Z axis is determined based on a vector pair constructed from the feature point pair;

[0122] and determining a scaling factor based on matched feature points between the target X-ray image and the local digitally reconstructed radiographic image, and determining a translation amount along the Z axis based on the scaling factor and a distance between the local digitally reconstructed radiographic image and the radiation source;

[0123] and determining a pixel movement distance coefficient corresponding to each unit physical distance movement based on a template image with a known physical translation relationship, and determining a translation amount along the X axis and a translation amount along the Y axis according to the pixel movement distance coefficient and a scaling factor;

[0124] The rotation angle around the X axis, the rotation angle around the Y axis, the rotation angle around the Z axis, the translation along the X axis, the translation along the Y axis, and the translation along the Z axis are used as the initial rigid registration solution.

[0125] In one embodiment, the initial solution determination module 506 is specifically configured to:

[0126] The three feature points in the target X-ray image are recorded as the first feature point, the second feature point and the third feature point, and the feature points determined to match them in the local digitally reconstructed radiographic image are recorded as the fourth feature point, the fifth feature point and the sixth feature point;

[0127] Constructing a first vector pointing from the first feature point to the second feature point, a second vector pointing from the first feature point to the third feature point, a third vector pointing from the fourth feature point to the fifth feature point, and a fourth vector pointing from the fourth feature point to the sixth feature point;

[0128] Determine an angle deviation between the angle between the first vector and the third vector and the angle between the second vector and the fourth vector;

[0129] Continue to extract three new feature points from the target X-ray image to obtain a new angle deviation value, and take the average of all angle deviation values ​​as the rotation angle around the Z axis.

[0130] In one embodiment, the initial solution determination module 506 is specifically configured to:

[0131] Any feature point in the target X-ray image is recorded as the seventh feature point, and the feature point determined to match it in the local digitally reconstructed radiographic image is recorded as the eighth feature point;

[0132] determining a first length value of a line connecting the seventh characteristic point and the eighth characteristic point in the target X-ray image and a second length value in the local digitally reconstructed radiographic image, respectively, to obtain a length ratio between the first length value and the second length value;

[0133] New feature points are continuously extracted from the target X-ray image to obtain new length ratios, and the average of all length ratios is used as the scaling factor.

[0134] In one embodiment, the adaptive fine-tuning module 508 is specifically configured to:

[0135] generating a new digitally reconstructed radiographic image corresponding to the CT image to be registered based on the initial rigid registration solution, and determining the similarity between the new digitally reconstructed radiographic image and the target X-ray;

[0136] If the similarity between the new digitally reconstructed radiographic image and the target X-ray does not meet a preset similarity threshold, determining a parameter fine-tuning range based on the similarity between the new digitally reconstructed radiographic image and the target X-ray;

[0137] Adaptively fine-tune the initial rigid registration solution according to the parameter fine-tuning range through each parameter fine-tuning model in the parameter fine-tuning model set, and obtain a subset of rigid registration solutions output by each parameter fine-tuning model;

[0138] According to the fitness value of each rigid registration solution contained in each rigid registration solution subset, a plurality of rigid registration solutions are screened from each rigid registration solution subset to construct a rigid registration solution set;

[0139] Determine the allocation weight corresponding to each parameter fine-tuning model based on the distribution of the rigid registration solution subset output by each parameter fine-tuning model in the rigid registration solution set;

[0140] According to the assigned weight, each rigid registration solution contained in the rigid registration solution set is assigned to each parameter fine-tuning model, so that each parameter fine-tuning model continues to perform adaptive fine-tuning operations on the assigned rigid registration solution according to the parameter fine-tuning range until the optimal rigid registration solution in the rigid registration solution set is determined when the preset conditions are met.

[0141] In one embodiment, the adaptive fine-tuning module 508 is specifically configured to:

[0142] Determining new adaptive fine-tuning control parameters based on the fine-tuning operators included in the fine-tuning parameter set and the adaptive fine-tuning control parameters used when performing the adaptive fine-tuning operation on the previous rigid registration solution;

[0143] determining new fine-tuning parameters according to the new adaptive fine-tuning control parameters;

[0144] The new fine-tuning parameters are used to perform an adaptive fine-tuning operation on the current rigid registration solution, and the rigid registration solution after the adaptive fine-tuning operation falls within the parameter fine-tuning range.

[0145] In one embodiment, the adaptive fine-tuning module 508 is further configured to:

[0146] When the fitness value of the new rigid registration solution obtained by the adaptive fine-tuning operation is higher than the fitness value of the current rigid registration solution, the new fine-tuning parameters are saved in the fine-tuning parameter set.

[0147] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those in the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference can be made to the corresponding content in the aforementioned method embodiment.

[0148] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned embodiments.

[0149] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 60, a memory 61, a bus 62 and a communication interface 63, wherein the processor 60, the communication interface 63 and the memory 61 are connected via the bus 62; the processor 60 is used to execute an executable module stored in the memory 61, such as a computer program.

[0150] The memory 61 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. The system network element communicates with at least one other network element via at least one communication interface 63 (which may be wired or wireless), and may utilize the Internet, a wide area network, a local area network, a metropolitan area network, or the like.

[0151] The bus 62 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0152] Among them, the memory 61 is used to store programs, and the processor 60 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 60 or implemented by the processor 60.

[0153] The processor 60 may be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above method may be performed by hardware integrated logic circuits or software instructions within the processor 60. The processor 60 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention may be directly executed by a hardware decoding processor or by a combination of hardware and software modules within the decoding processor. The software modules may be located in storage media well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or the like. The storage medium is located in the memory 61 , and the processor 60 reads the information in the memory 61 and completes the steps of the above method in combination with its hardware.

[0154] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be referred to the previous method embodiment and will not be repeated here.

[0155] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0156] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for solving rigid registration parameters, characterized in that: include: Obtain the CT image to be registered and the target X-ray image; generating a plurality of digitally reconstructed radiographic images corresponding to the CT image to be registered based on a plurality of preset rigid registration partial solutions, and screening out a target digitally reconstructed radiographic image that matches the target X-ray image; wherein the rigid registration partial solutions include a rotation angle around the X-axis and a rotation angle around the Y-axis; Acquire a local digitally reconstructed radiographic image according to the target X-ray image, the target digitally reconstructed radiographic image, and the corresponding rigid registration partial solution, and determine an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image; Adaptively fine-tuning the initial rigid registration solution to obtain a rigid registration solution set, and determining an optimal rigid registration solution in the rigid registration solution set until a preset condition is satisfied, including: generating a new digitally reconstructed radiographic image corresponding to the CT image to be registered based on the initial rigid registration solution, and determining a similarity between the new digitally reconstructed radiographic image and the target X-ray; If the similarity between the new digitally reconstructed radiographic image and the target X-ray does not meet a preset similarity threshold, determining a parameter fine-tuning range based on the similarity between the new digitally reconstructed radiographic image and the target X-ray; Performing an adaptive fine-tuning operation on the initial rigid registration solution according to the parameter fine-tuning range by each parameter fine-tuning model in the parameter fine-tuning model set, to obtain a subset of rigid registration solutions output by each parameter fine-tuning model; screening a plurality of rigid registration solutions from each of the rigid registration solution subsets according to the fitness value of each rigid registration solution contained in each of the rigid registration solution subsets to construct a rigid registration solution set; Determining an allocation weight corresponding to each of the parameter fine-tuning models based on a distribution of the rigid registration solution subsets output by each of the parameter fine-tuning models in the rigid registration solution set; According to the assigned weight, each rigid registration solution contained in the rigid registration solution set is assigned to each parameter fine-tuning model, so that each parameter fine-tuning model continues to perform adaptive fine-tuning operations on the rigid registration solution assigned thereto according to the parameter fine-tuning range until a preset condition is met, and the parameters at this time are determined to be the optimal rigid registration solution in the rigid registration solution set; wherein the preset condition is: the similarity between the new digitally reconstructed radiographic image generated based on the optimal rigid registration solution and the target X-ray image reaches a preset similarity threshold.

2. The method for solving rigid registration parameters according to claim 1, wherein: Determining an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image includes: For any three feature points in the target X-ray image, determining feature points that match the three feature points from the local digitally reconstructed radiographic image to form a feature point pair, and determining a rotation angle around the Z axis based on a vector pair constructed from the feature point pair; and determining a scaling factor based on feature points matched between the target X-ray image and the local digitally reconstructed radiographic image, and determining a translation amount along the Z axis based on the scaling factor and a distance between the local digitally reconstructed radiographic image and a radiation source; and determining a pixel movement distance coefficient corresponding to each unit physical distance movement based on a template image with a known physical translation relationship, and determining a translation amount along the X axis and a translation amount along the Y axis according to the pixel movement distance coefficient and the scaling factor; The rotation angle around the X axis, the rotation angle around the Y axis, the rotation angle around the Z axis, the translation along the X axis, the translation along the Y axis, and the translation along the Z axis are used as an initial rigid registration solution.

3. The method for solving rigid registration parameters according to claim 2, wherein: For any three feature points in the target X-ray image, determining feature points matching the three feature points from the local digitally reconstructed radiographic image to form a feature point pair, and determining a rotation angle around the Z axis based on a vector pair constructed from the feature point pair, including: Recording the three feature points in the target X-ray image as a first feature point, a second feature point, and a third feature point, and recording the feature points determined to match them in the local digitally reconstructed radiographic image as a fourth feature point, a fifth feature point, and a sixth feature point; Constructing a first vector pointing from the first feature point to the second feature point, a second vector pointing from the first feature point to the third feature point, a third vector pointing from the fourth feature point to the fifth feature point, and a fourth vector pointing from the fourth feature point to the sixth feature point; Determine an angle deviation between an angle between the first vector and the third vector and an angle between the second vector and the fourth vector; Continue to extract three new feature points from the target X-ray image to obtain a new angle deviation value, and use the average of all the angle deviation values ​​as the rotation angle around the Z axis.

4. The method for solving rigid registration parameters according to claim 2, wherein: Determining a scaling factor based on matched feature points between the target X-ray image and the local digitally reconstructed radiographic image includes: Recording any feature point in the target X-ray image as the seventh feature point, and recording the feature point determined to match it in the local digitally reconstructed radiographic image as the eighth feature point; determining a first length value of a line connecting the seventh feature point and the eighth feature point in the target X-ray image and a second length value in the local digitally reconstructed radiographic image, respectively, to obtain a length ratio between the first length value and the second length value; New feature points are continuously extracted from the target X-ray image to obtain new length values, and the average of all the length ratios is used as a scaling factor.

5. The method for solving rigid registration parameters according to claim 1, wherein: By means of each parameter fine-tuning model, the adaptive fine-tuning operation is continued on the rigid registration solution assigned thereto according to the parameter fine-tuning range, including: Determining new adaptive fine-tuning control parameters based on the fine-tuning operator included in the fine-tuning parameter set and the adaptive fine-tuning control parameters used when performing the adaptive fine-tuning operation on the previous rigid registration solution; Determining new fine-tuning parameters according to new adaptive fine-tuning control parameters; An adaptive fine-tuning operation is performed on the current rigid registration solution using the new fine-tuning parameters, so that the rigid registration solution after the adaptive fine-tuning operation falls within the parameter fine-tuning range.

6. The method for solving rigid registration parameters according to claim 5, characterized in that: The method further comprises: When the fitness value of the new rigid registration solution obtained by the adaptive fine-tuning operation is higher than the fitness value of the current rigid registration solution, the new fine-tuning parameters are saved in the fine-tuning parameter set.

7. A rigid registration parameter solving device, characterized in that: include: An image acquisition module is used to acquire the CT image to be registered and the target X-ray image; an image matching module, configured to generate a plurality of digitally reconstructed radiographic images corresponding to the CT image to be registered based on a plurality of preset rigid registration partial solutions, and to select a target digitally reconstructed radiographic image that matches the target X-ray image; wherein the rigid registration partial solutions include a rotation angle around the X-axis and a rotation angle around the Y-axis; an initial solution determination module, configured to obtain a local digitally reconstructed radiographic image based on the target X-ray image, the target digitally reconstructed radiographic image, and the corresponding rigid registration partial solution, and determine an initial rigid registration solution based on the local digitally reconstructed radiographic image and the target X-ray image; An adaptive fine-tuning module is configured to perform an adaptive fine-tuning operation on the initial rigid registration solution to obtain a rigid registration solution set, and determine an optimal rigid registration solution in the rigid registration solution set when a preset condition is met, including: generating a new digitally reconstructed radiographic image corresponding to the CT image to be registered based on the initial rigid registration solution, and determining a similarity between the new digitally reconstructed radiographic image and the target X-ray; If the similarity between the new digitally reconstructed radiographic image and the target X-ray does not meet a preset similarity threshold, determining a parameter fine-tuning range based on the similarity between the new digitally reconstructed radiographic image and the target X-ray; Performing an adaptive fine-tuning operation on the initial rigid registration solution according to the parameter fine-tuning range by each parameter fine-tuning model in the parameter fine-tuning model set, to obtain a subset of rigid registration solutions output by each parameter fine-tuning model; screening a plurality of rigid registration solutions from each of the rigid registration solution subsets according to the fitness value of each rigid registration solution contained in each of the rigid registration solution subsets to construct a rigid registration solution set; Determining an allocation weight corresponding to each of the parameter fine-tuning models based on a distribution of the rigid registration solution subsets output by each of the parameter fine-tuning models in the rigid registration solution set; According to the assigned weight, each rigid registration solution contained in the rigid registration solution set is assigned to each parameter fine-tuning model, so that each parameter fine-tuning model continues to perform adaptive fine-tuning operations on the rigid registration solution assigned thereto according to the parameter fine-tuning range until a preset condition is met, and the parameters at this time are determined to be the optimal rigid registration solution in the rigid registration solution set; wherein the preset condition is: the similarity between the new digitally reconstructed radiographic image generated based on the optimal rigid registration solution and the target X-ray image reaches a preset similarity threshold.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 6.

Images