Image registration method based on historical record fast differential evolution sampling consensus algorithm

Abstract

The invention discloses an image registration method based on a historical record fast differential evolution sampling consensus algorithm. The method comprises the following steps: 1, image acquisition: acquiring an image to be registered and a reference image; 2, feature extraction: respectively extracting point features of the to-be-registered image and the reference image by adopting an SIFT (Scale Invariant Feature Transform) algorithm; 3, feature matching: performing feature rough matching on the feature set of the points to be registered and the feature set of the reference points to obtain a sample set; 4, setting parameters; 5, sorting the sample set in an ascending order, and initializing a sample inner point file AS and a model inner point file AM; 6, initializing the population for the first time; 7, updating a rapid inspection execution threshold value uT; 8, a variation stage; 9, a selection stage; 10, updating mutation operator parameters; 11, performing resident initialization on the population; 12, repeating the steps 7-11; and 13, selecting an optimal individual in the population to carry out spatial transformation on the to-be-registered image to obtain a registered image. The method has the advantages that the algorithm operation speed can be increased under the condition that the image registration precision is guaranteed.

Description

Technical Field

[0001] The invention relates to the technical field of image registration, and in particular to an image registration method based on a historical record fast differential evolution sampling consensus algorithm. Background Art

[0002] Image registration is a basic technique in image processing, which aims to superimpose and fuse images of similar scenes containing common areas to generate information-rich images for subsequent processing or decision-making. It is widely used in many fields and is the core technology for image analysis tasks in remote sensing, medicine, computer vision and other fields.

[0003] Image registration methods mainly include three categories: image region-based registration methods, image feature-based registration methods, and deep learning-based registration methods. Image feature-based registration methods solve the transformation model between images by extracting robust features from the image. It is fast and efficient, and is insensitive to image changes. Image features can be divided into three categories: point features, line features, and surface features. Among them, the scale-invariant feature transform SIFT operator is widely used due to its excellent performance. The registration method based on image point features includes four steps: feature point extraction, feature point matching, transformation model solution, and image fusion. The task of the transformation model solution step is to perform robust estimation, that is, to estimate the model parameters from samples contaminated by noise. When a sample is severely contaminated by noise, it is a sample that does not conform to the hypothesized model and is called an outlier (outlier), otherwise it is called an inlier. Since there are a large number of outliers after the feature point matching stage, the challenge in the transformation model solution stage is how to robustly return a consensus set with a large number of inliers.

[0004] In order to find the correct match between two images, robust estimation is required. The purpose of robust estimation is to recover the geometric model from erroneous observation data. Commonly used robust estimation algorithms include random sampling consensus algorithm RANSAC, M-estimators, truncated least squares, weighted q-norm estimation, etc. The RANSAC algorithm proposed by Fischler et al. has become a widely used robust estimation algorithm in the field of computer vision due to its simple implementation and efficient execution. It can effectively and robustly solve the transformation model between images from a set of observations with different errors, even if the sample set is a sample set with a rate of more than 50% outliers. Compared with other robust estimators, RANSAC is robust, efficient and simple.

[0005] The differential evolution sampling consensus algorithm DESAC is a random sample consensus algorithm RANSAC that directly optimizes the transformation model by applying the differential evolution algorithm. It has strong internal point search ability and robustness. However, the time performance of the DESAC algorithm is affected by three factors: the number of verifications for each individual, the population size, and the number of population iterations. The large total number of verifications (total number of verifications = number of verifications for each individual × population size × number of population iterations) leads to poor time performance of the DESAC algorithm, which does not meet the speed requirements of point feature image registration. The specific defects of the DESAC algorithm in terms of time performance are as follows:

[0006] (1) The cost of verifying each hypothesis model is high. When performing full verification of a hypothesis model, if the total number of samples is large, the running cost of each full verification will be high.

[0007] (2) The computational overhead of resident initialization is high and the benefits are low. During the resident initialization phase, the DESAC algorithm will initialize individuals in batches and compare and replace them with random individuals in the population. However, most of the replacements are unsuccessful, resulting in high computational overhead and low benefits.

[0008] (3) The fixed scaling factor F results in inefficient exploration and development of the population.

[0009] Disadvantages of existing technology: The existing differential evolution sampling consensus algorithm DESAC takes a long time and has low efficiency when performing image registration tasks. Summary of the invention

[0010] The present invention provides an image registration method based on a historical record rapid differential evolution sampling consensus algorithm, which can improve the algorithm running speed while ensuring the image registration accuracy.

[0011] To achieve the above object, the present invention provides an image registration method based on a historical record fast differential evolution sampling consensus algorithm, the key of which is to include the following steps:

[0012] Step 1: Image acquisition: The image acquisition module acquires an image to be registered I1(x, y) and a reference image I2(x′, y′), and passes them to the feature extraction module;

[0013] Step 2: Feature extraction: The feature extraction module uses the SIFT algorithm to extract the point features of the image to be registered I1 (x, y) and the reference image I2 (x′, y′), respectively, to obtain the feature set F1 of the point to be registered and the feature set F2 of the reference point, and pass them to the feature matching module;

[0014] Step 3: Feature matching: The feature matching module performs rough feature matching on the feature set F1 of the points to be registered and the feature set F2 of the reference points to obtain a set of matching point pairs, i.e., a sample set S, and passes the sample set S to the image registration module of the fast differential evolution sampling consensus algorithm FDESAC based on historical records; F1 = {f1, f2, …, f a}, F2={f′1,f′2,…,f′ b}, S={s1,s2,…,s n}, where a represents the number of features of the points to be registered, b represents the number of features of the reference points, and n represents the number of matching point pairs;

[0015] Step 4: Parameter setting: The image registration module sets parameters according to the sample set S: SIFT feature distance ratio D r , error threshold t for determining internal points, population size NP, weight factor mean μw, pre-test sampling number N f Set up and set the neighborhood range Nr = NP / 10, the initial number of internal points n f =N f ;

[0016] Step 5: The image registration module registers the sample set S according to the SIFT feature distance ratio D r Sort in ascending order, then calculate BP based on the sample set S, and initialize the sample internal point file A S and model interior point file A M Among them, A S ={a s,1 ,a s,2 ,…,a s,n}, each sample point records a s,i Stored samples i The number of times it becomes an interior point, A M ={a m,1 ,a m,2 ,…,a m,n}, model internal point record a m,i Stored samples i The total number of internal points in the historical population involved in the construction;

[0017] According to the pre-test sampling number N f and the initial number of interior points n f Calculate the fast test execution threshold uT;

[0018] Step 6: Initialize the population for the first time: Randomly select m samples from the first BP*|S| range of the sample set S to initialize the individual X i , through the test function Test for individual X i, i∈[0,NP] to test, and the test results are i Evaluation index EI i , and update the model internal point file A M ;

[0019] Step 7: According to the pre-test sampling number N f And the number of inner points Inlier(X) updates the fast test execution threshold uT;

[0020] Step 8: Mutation phase: Initialize the weight coefficient set S of the mutant individuals that successfully enter the next round of iteration w , according to the weight coefficient set S w Calculate the scaling factor F i , according to the scaling factor F i Calculate the variant individual V i ;

[0021] Step 9: Selection phase: Use the test function Test to test the variant individual V i Test and get the variant individual V i Evaluation index EI y ;

[0022] Individual X i Evaluation index EI i With the variant individual V i Evaluation index EI y For comparison, if the variant individual V i Evaluation index EI y Better than individual X i Evaluation index EI i , then the individual X i Replaced with variant individual V i , the mutant individual V i The weight factor w is recorded in the weight coefficient set S w middle;

[0023] Otherwise, randomly select m samples from the first BP*|S| range of the sample set S to reinitialize individual X i , through the test function Test for individual X i Test and get individual X i Evaluation index EI i , and update the model internal point file A M ;

[0024] Step 10: Update mutation operator parameters: Calculate weight coefficient set S w The mean avg of w The mean avg of the weight factor is calculated as the mean μw;

[0025] Step 11: Resident initialization population: Calculate the number of resident initialization repetitions r, if the mutation is successful, the number of individuals | S w | is equal to 0, then r = 3*NP; otherwise, r = 3*NP / |S w |;

[0026] Find the worst individual X in the population worst ;

[0027] If iter>N f And iter%uT≠0, iter represents the number of iterations, randomly select m model internal point records a in the first BP*|S| range of the sample set S m ≠0 samples reinitialize individual X new ; Otherwise, randomly select m samples from the first BP*|S| range of the sample set S to reinitialize individual X new ;

[0028] Through the test function Test, the individual X new Test and get individual X new Evaluation index EI new , and update the model internal point file A M ;

[0029] Individual X new Evaluation index EI new and the worst individual X worst Evaluation index EI worst For comparison, if individual X new Evaluation index EI new Better than the worst individual X worst Evaluation index EI worst , then the worst individual X worst Replace with individual X new ;

[0030] Step 12: Repeat steps 7-11 until the preset number of iterations is reached, then proceed to step 13;

[0031] Step 13: Select the best individual in the population: Select the individual with the best evaluation index EI from the population and save it to the best individual X Best , the optimal individual X Best As a spatial geometric transformation model, the image to be registered I1 (x, y) is spatially transformed to obtain a registered image.

[0032] Through the above design, firstly, the historical records of population iteration are used to skip the poor samples when verifying the hypothesis model; secondly, the better samples are selected when constructing the new hypothesis model in the resident initialization to avoid wasting resources to generate useless hypothesis models; finally, the proposed adaptive scaling factor strategy is used to enable the algorithm to focus more on population development and accelerate the convergence of the algorithm. In this way, the algorithm's running speed and efficiency are effectively improved while ensuring the accuracy of image registration.

[0033] In the stage of verifying the hypothesis model, a fast verification strategy based on the historical records of sample inliers is proposed. By recording the total number of times each sample becomes an inlier, the samples with a high probability of being outliers are skipped, and the verification of them is omitted and they are directly marked as outliers, which greatly shortens the running time of the verification algorithm.

[0034] In the resident initialization stage, a resident initialization strategy based on the historical records of model interior points is proposed. It records the total number of interior points of the hypothetical models that each sample has participated in constructing in history, and uses it to avoid selecting poor samples to generate hypothetical models, thereby improving the efficiency of the algorithm. At the same time, the individuals compared and replaced by the resident initialization are improved from random individuals to the worst individuals in the population, which improves the effectiveness of the resident initialization and accelerates the convergence of the algorithm.

[0035] In the mutation stage, the scaling factor F determines the variation step length of the differential evolution algorithm and affects the global optimization ability of the algorithm. In order to reduce the number of iterations of the population and keep the accuracy almost unchanged, and improve the development weight of the differential evolution algorithm, a scaling factor adaptive strategy based on the number of successful mutations is proposed. The improvement idea of ​​the scaling factor adaptive strategy is as follows:

[0036] First, in order to increase the diversity of the scaling factor F value, it is necessary to introduce a random function to give each individual a different basic F value. Second, in order to balance accuracy and speed, the basic value of F needs to be maintained in an appropriate range. Third, when the population development in a certain generation is strong, the F value needs to be increased to avoid falling into the local optimum; when the exploration is strong, the F value should be appropriately lowered to speed up the convergence of the population. Therefore, in order to judge the exploration and development of the population in a certain generation, it is necessary to associate F with the number of successful mutations of the previous generation of the population. Fourth, in order to further avoid the population converging to the local optimum and causing a significant decrease in accuracy, it is necessary to slightly increase the F value of individuals that successfully mutated in the previous generation.

[0037] In summary, the proposed scaling factor adaptive strategy only needs to increase the F value at the right time, that is, when the population development situation is strong, and on the right individuals, that is, individuals that have successfully mutated in the previous generation, to ensure that the accuracy is only slightly reduced, while setting the rest to a lower F value can significantly improve the running speed of the algorithm.

[0038] Preferably, in step 2, the SIFT algorithm extracts point features in spaces of various scales, including but not limited to corner points, edge points, bright spots in dark areas, and dark spots in bright areas.

[0039] By extracting point features, the amount of information in the image can be greatly compressed. The feature extraction speed is fast and it is robust to changes in image grayscale. The SIFT algorithm extracts feature points in spaces of different scales. The rotation and scaling of the image will not affect the correctness of the feature points. The features extracted by the SIFT algorithm are 128-dimensional one-dimensional vectors, which are very stable local features.

[0040] Preferably, in step 5, the expression of BP is calculated according to the sample set S as follows:

[0041]

[0042] Among them, m is the minimum number of samples for generating the model, |S| is the total number of samples, and BP represents the proportion of excellent samples in the sample set S;

[0043] The expression for calculating the fast test execution threshold uT is as follows:

[0044]

[0045] In the above formula, for example, the predicted inlier rate (n f / N f )×100%=40%, uT=13, which means that one generation of full test is performed after every 12 generations of fast tests; and if the predicted inlier rate is 5%, uT=6, which means that one generation of full test is performed after every 5 generations of fast tests.

[0046] The setting of the threshold uT is crucial. When the threshold uT increases, the proportion of fast tests in all types of tests will increase, allowing the algorithm to verify at a very fast speed in the middle and late stages. However, the disadvantage is that when the inlier rate is low and the number of samples is large, it is easy to miss the real inliers, making the consistent set returned by the algorithm incomplete. When the threshold uT decreases, the proportion of fast tests in all types of tests will decrease, and the algorithm will frequently verify all samples to ensure that there are no omissions in the returned inliers, but the disadvantage is that the proportion of fast tests is small and the acceleration effect is not excellent. Therefore, it is necessary to dynamically adjust the threshold uT so that it increases with the predicted inlier rate (nT) of the FDESAC algorithm. f / N f )×100% dynamic changes.

[0047] As a preference: in step 8, calculate the scaling factor F i The expression is as follows:

[0048]

[0049] Among them, rand(0.6,0.75) is a uniformly distributed random number with a value between (0.6,0.75), |S w | is the number of successfully mutated individuals in the previous generation of the population, NP is the population size, and g represents the gth generation;

[0050] Calculate the variant individual V i The expression is as follows:

[0051] V i =wV rand1 +(1-w)V local_best

[0052]

[0053] Among them, the i-th variant individual V i By randomly mutating individual V rand1 and the optimal mutation individual V in the neighborhood local_best The weighted value is obtained by weighting factor w, w∈[0,1]; F is the scaling factor, X local_best Is individual X i The best neighbor within the index range of .

[0054] Preferably, in step 10, the expression for calculating the mean value μw of the weight factor is as follows:

[0055] μw=(1-0.1)*μw+0.1*avg.

[0056] As a preferred embodiment: in step 12, the entire iterative process is divided into an archive recording phase and an archive application phase, and the dividing point between the two phases is the maximum sample in-point record max(a s )=NP;

[0057] For the archival recording stage, the test function Test is used to test the individual X i The inspection method is pre-inspection + full inspection;

[0058] In the archive application stage, the test function Test is used to test the individual X i The inspection method is pre-inspection + rapid inspection;

[0059] When performing a quick test, if sample s i The sample point record a s,i If it is 0, then the verification is skipped and it is directly judged as an outlier;

[0060] In the archive application stage, when the population iteration number iter%uT=0, the pre-check + quick check is not performed, but the pre-check + full check is performed instead.

[0061] There is no lack of such a small probability situation in the rapid test: after a sample has passed the archive recording stage, its sample inlier record is still 0, but this sample is actually an inlier that meets the conditions. In order to avoid the real inlier being missed in the archive application stage, the fast differential evolution sampling consensus algorithm FDESAC based on historical records sets a threshold uT. When the population iteration number iter%uT=0, the pre-test + rapid test is not performed, and the pre-test + full test is performed instead, so that it can timely detect whether there are missed inliers that are skipped by the rapid test.

[0062] As a preference: the model internal point file A M Update when extracting samples to generate the hypothesis model, that is, initializing the population (including the first population initialization, resident initialization, and selection phase);

[0063] First, the algorithm randomly selects m samples from the excellent samples in the range of the first BP*100% to construct the minimum sample subset S min ={s x1 ,s x2 ,…,s xm}, where m is the minimum number of samples for building the model; then, using S min Generate a hypothesis model X and verify it to obtain the number of internal points Inlier(X); finally, for each sample s involved in building the hypothesis model X xi , the corresponding model internal point record a m,xi The update expression is as follows:

[0064] a m,xi =a m,xi +Inlier(X).

[0065] As a preference: in step 4, the SIFT feature distance ratio D r is set to 0.8, and the error threshold t for determining the internal point is set to 5 pixels.

[0066] For the input sample set S, to ensure that enough internal points are obtained, the SIFT feature distance ratio D r It is set to 0.8. Since the image registration of this dataset is difficult, the error threshold t for determining the inliers is increased to 5 pixels to increase the error tolerance for determining the inliers.

[0067] Preferably, the evaluation index EI includes two indicators: the number of inliers and the inlier error.

[0068] When comparing and selecting individuals, priority is given to those with more inliers in the first indicator. If the number of inliers is the same, the one with lower error in the second indicator is selected.

[0069] Beneficial effects of the present invention:

[0070] 1. In the stage of verifying the hypothesis model, a fast verification strategy based on the historical records of sample inliers is proposed. By recording the total number of times each sample becomes an inlier, samples with a high probability of being outliers are skipped, and the verification of them is omitted and they are directly marked as outliers, which greatly shortens the running time of the verification algorithm.

[0071] 2. In the resident initialization stage, a resident initialization strategy based on the historical records of model interior points is proposed. It records the total number of interior points of the hypothetical models that each sample has participated in constructing in history, and uses it to avoid selecting those poor samples to generate hypothetical models, thereby improving the efficiency of the algorithm. At the same time, the individuals compared and replaced by the resident initialization are improved from random individuals to the worst individuals in the population, which improves the effectiveness of the resident initialization and accelerates the convergence of the algorithm.

[0072] 3. In the mutation stage, the scaling factor F determines the variation duration of the differential evolution algorithm and affects the global optimization ability of the algorithm. In order to reduce the number of iterations of the population and keep the accuracy almost unchanged, and improve the development weight of the differential evolution algorithm, an adaptive scaling factor strategy based on the number of successful mutations is proposed. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 It is a schematic diagram of the process flow of the fast differential evolution sampling consensus algorithm based on historical records in the present invention;

[0074] Figure 2 It is a schematic diagram of updating the in-sample point archive;

[0075] Figure 3 Schematic diagram of the rapid inspection strategy;

[0076] Figure 4 This is a schematic diagram for permanent initialization;

[0077] Figure 5 The speedup diagram for different image pairs;

[0078] Figure 6 Comparison chart of total average number of inliers and total maximum number of inliers for different algorithms;

[0079] Figure 7 This is the comparison chart of the total RMSE reprojection error;

[0080] Figure 8 It is the comparison chart of total average inlier error;

[0081] Fig. 9 It is a line graph showing the average number of inliers and the maximum number of inliers as a function of the number of verifications;

[0082] Fig.10 This is a curve diagram showing the effect of setting different F values ​​on FDESAC; DETAILED DESCRIPTION

[0083] The present invention is further described in detail below in conjunction with the accompanying drawings and specific examples. The following examples or drawings are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0084] like Figure 1 As shown: An image registration method based on a historical record fast differential evolution sampling consensus algorithm comprises the following steps:

[0085] Step 1: Image acquisition: The image acquisition module acquires an image to be registered I1(x, y) and a reference image I2(x′, y′), and passes them to the feature extraction module;

[0086] Step 2: Feature extraction: The feature extraction module uses the SIFT algorithm to extract the point features of the image to be registered I1 (x, y) and the reference image I2 (x′, y′), respectively, to obtain the feature set F1 of the point to be registered and the feature set F2 of the reference point, and pass them to the feature matching module;

[0087] Step 3: Feature matching: The feature matching module performs rough feature matching on the feature set F1 of the points to be registered and the feature set F2 of the reference points to obtain a set of matching point pairs, i.e., a sample set S, and passes the sample set S to the image registration module of the fast differential evolution sampling consensus algorithm FDESAC based on historical records;

[0088] Step 4: Parameter setting: The image registration module sets parameters according to the sample set S: SIFT feature distance ratio D r , error threshold t for determining internal points, population size NP, weight factor mean μw, pre-test sampling number N f Set up and set the neighborhood range Nr = NP / 10, the initial number of internal points n f =N f ;

[0089] Step 5: The image registration module registers the sample set S according to the SIFT feature distance ratio D r Sort in ascending order, then calculate BP based on the sample set S, and initialize the sample internal point file A S and model interior point file A M ;

[0090]

[0091] Among them, m is the minimum number of samples to generate the model, |S| is the total number of samples;

[0092] According to the pre-test sampling number N f and the initial number of interior points n fCalculate the fast test execution threshold uT, the expression is as follows:

[0093]

[0094] Step 6: Initialize the population for the first time: Randomly select m samples from the first BP*|S| range of the sample set S to initialize the individual X i , through the test function Test for individual X i , i∈[0,NP] to test, and the test results are i Evaluation index EI i , and update the model internal point file A M ;

[0095] Step 7: According to the pre-test sampling number N f And the number of inner points Inlier(X) updates the fast test execution threshold uT;

[0096] Step 8: Mutation phase: Initialize the weight coefficient set S of the mutant individuals that successfully enter the next round of iteration w , according to the weight coefficient set S w Calculate the scaling factor F i , according to the scaling factor F i Calculate the variant individual V i ;

[0097] Calculate the scaling factor F i The expression is as follows:

[0098]

[0099] Among them, rand(0.6,0.75) is a uniformly distributed random number with a value between (0.6,0.75), |S w | is the number of successfully mutated individuals in the previous generation of the population, NP is the population size, and g represents the gth generation;

[0100] Calculate the variant individual V i The expression is as follows:

[0101] V i =wV rand1 +(1-w)V locla_best

[0102]

[0103] Among them, the i-th variant individual V i By randomly mutating individual V rand1 and the optimal mutation individual V in the neighborhood local_best The weighted value is obtained by weighting factor w, w∈[0,1]; F is the scaling factor, X local_bestIs individual X i The best neighbor within the index range of ;

[0104] Step 9: Selection phase: Use the test function Test to test the variant individual V i Test and get the variant individual V i Evaluation index EI y ;

[0105] Individual X i Evaluation index EI i With the variant individual V i Evaluation index EI y For comparison, if the variant individual V i Evaluation index EI y Better than individual X i Evaluation index EI i , then the individual X i Replaced with variant individual V i , the mutant individual V i The weight factor w is recorded in the weight coefficient set S w middle;

[0106] Otherwise, randomly select m samples from the first BP*|S| range of the sample set S to reinitialize individual X i , through the test function Test for individual X i Test and get individual X i Evaluation index EI i , and update the model internal point file A M ;

[0107] Step 10: Update mutation operator parameters: Calculate weight coefficient set S w The mean avg of w The mean avg of the weight factor is calculated as μw, and the expression is as follows:

[0108] μw=(1-0.1)*μw+0.1*avg

[0109] Step 11: Resident initialization population: Calculate the number of resident initialization repetitions r, if the mutation is successful, the number of individuals | S w | is equal to 0, then r = 3*NP; otherwise, r = 3*NP / |S w |;

[0110] Find the worst individual X in the population worst ;

[0111] If iter>N fAnd iter%uT≠0, iter represents the number of iterations, randomly select m model internal point records a in the first BP*|S| range of the sample set S m ≠0 samples reinitialize individual X new ; Otherwise, randomly select m samples from the first BP*|S| range of the sample set S to reinitialize individual X new ;

[0112] Through the test function Test, the individual X new Test and get individual X new Evaluation index EI new , and update the model internal point file A M ;

[0113] Individual X new Evaluation index EI new and the worst individual X worst Evaluation index EI worst For comparison, if individual X new Evaluation index EI new Better than the worst individual X worst Evaluation index EI worst , then the worst individual X worst Replace with individual X new ;

[0114] Step 12: Repeat steps 7-11 until the preset number of iterations is reached, then proceed to step 13;

[0115] Step 13: Select the best individual in the population: Select the individual with the best evaluation index EI from the population and save it to the best individual X Best , the optimal individual X Best As a spatial geometric transformation model, the image to be registered I1 (x, y) is spatially transformed to obtain a registered image.

[0116] Furthermore, in step 2, the SIFT algorithm extracts point features in various scales of space, including but not limited to corner points, edge points, bright spots in dark areas, and dark spots in bright areas.

[0117] Setting and updating of sample internal point files: First, for all sample sets S = {s1, s2, ..., s n}, generate a sample internal point file {a s,1 ,a s,2 ,…,a s,n} corresponds to it one by one, and each sample point records a s,i Stored samples i The number of times it becomes an internal point. When the FDESAC algorithm needs to perform a verification operation on the hypothesis model, if the sample si It is judged as an interior point, and its corresponding sample interior point record is a s,i =a s,i +1, like Figure 2 shown.

[0118] In step 12, the entire iterative process is divided into an archive recording phase and an archive application phase, and the dividing point between the two phases is the maximum sample inlier record max(a s )=NP;

[0119] For the archival recording stage, the test function Test is used to test the individual X i The inspection method is pre-inspection + full inspection;

[0120] In the archive application stage, the test function Test is used to test the individual X i The inspection method is pre-inspection + rapid inspection;

[0121] When performing a quick test, if sample s i The sample point record a s,i If it is 0, then the verification is skipped and it is directly judged as an outlier;

[0122] In the archive application stage, when the population iteration number iter%uT=0, the pre-check + quick check is not performed, and the pre-check + full check is performed instead, such as Figure 3 shown.

[0123] The test function of the FDESAC algorithm is shown in Algorithm 4-1.

[0124]

[0125]

[0126] The model internal point file is also set for each sample. n}, set the model internal point file A M ={a m,1 ,a m,2 ,…,a m,n Model internal point record a m,i The total number of internal points of the historical hypothesis model constructed by the sample is stored.

[0127] The model internal point file A M When the sample is drawn to generate the hypothesis model, that is, the population is initialized, the algorithm first randomly selects m samples from the excellent samples in the previous BP*100% range to construct the minimum sample subset S min ={s x1 ,s x2,…,s xm}, where m is the minimum number of samples for building the model; then, using S min Generate a hypothesis model X and verify it to obtain the number of internal points Inlier(X); finally, for each sample s involved in building the hypothesis model X xi , the corresponding model internal point record a m,xi The update expression is as follows:

[0128] a m,xi =a m,xi +Inlier(X).

[0129] Similarly, model internal point file A M It also requires a certain amount of accumulation in the early stage before it can be applied. Its archival recording stage is shorter, and it is similar to the internal point number n. f The adaptive cycle is consistent, when the population iteration number iter> pre-test sampling number N f After that, you can enter the file application stage. After entering the file application stage, it is also executed at intervals, that is, every time uT-1 generation is used, 1 generation will be stopped.

[0130] The resident initialization process using the model internal point file is as follows:

[0131] (1) To initialize a new individual X new , when randomly selecting m samples, each sample s will be checked xi The corresponding model internal point record a m,xi Does it satisfy a m,xi ≠0. If it is not satisfied, a new sample is selected until the condition is satisfied. m,xi =0 means that the model generated by this sample is likely to be of poor quality, and replacing it will help improve the efficiency of the algorithm.

[0132] (2) For the individuals to be compared, we no longer randomly select them from the population, but select the worst individual X in the population. worst , X new With X worst Compare and retain the better one. It can improve the effectiveness of resident initialization and speed up the convergence of the algorithm.

[0133] (3) Repeat the steps (1) and (2). The schematic diagram of the permanent initialization is as follows: Figure 4 shown.

[0134] The complete pseudocode description of the FDESAC algorithm is shown in Algorithm 4-2.

[0135]

[0136]

[0137] Next, a comprehensive evaluation of the present invention is carried out through experiments.

[0138] Image dataset: Multiple images from an advanced homography estimation dataset are used to solve the consensus set and transformation matrix. The dataset provides multiple pairs of manually correct matching points to more accurately estimate the image registration accuracy.

[0139] Comparison method: In the advanced experiment, the fast differential evolution sampling consensus algorithm based on historical records in the present invention will be compared with the benchmark random sampling consensus algorithm (RANSAC), the fast random sampling consensus algorithm based on model parameter pre-verification (PERANSAC), the particle swarm optimization sampling consensus algorithm (PSOSAC), the adaptive genetic algorithm sampling consensus algorithm (AGASAC), the differential evolution algorithm with combined strategy (CSDE), the crowded differential evolution algorithm based on Voronoi neighborhood (VNCDE), and the differential evolution sampling consensus algorithm (DESAC). All experiments were run on an Intel quad-core CPU with 2.5GHz and 8GB memory.

[0140] Parameter setting: For the input sample set S, to ensure that enough internal points are obtained, the SIFT feature distance ratio D r Set to 0.8. Due to the difficulty of image registration in this dataset, the error threshold t for judging the inliers is increased to 5 pixels to increase the error tolerance for judging the inliers. Due to the randomness of the algorithm, except for the maximum number of inliers (maximum inlier rate), all experimental results are taken as the average after 100 repeated runs. For the comparison algorithms that have no special improvements on the population size NP and the scaling factor F, NP is set to 30 and F is set to 0.75. The common parameters of the FDESAC algorithm and the DESAC algorithm are consistent: the initial value μw of the mean of the weighted coefficient is set to 0.5, the neighborhood size Nr is set to NP / 10, and the number of pre-test samples N is set to 0.5. f Set to 15.

[0141] For the AGASAC algorithm, the power factor γ of the control function is set to 10, and the power factor δ of the controlled Gaussian function is set to 1. Due to the large number of iterations of the AGASAC algorithm, a high learning rate η will cause the probability of a certain sample being selected to accumulate too high, resulting in repeated extraction errors when extracting the minimum sample set. Therefore, η is flexibly set to the inverse of the total number of samples |S|, and when there is a sample with η greater than 0.2, the η of all samples is reset. For the PSOSAC algorithm, the learning rate weights c1 and c2 and the inertia weight ω are set to 0.5. For the DIDE algorithm, the upper and lower bounds of the translation dimensions in the perspective matrix are set to ±1000, and the other dimensions are set to ±2. Due to its mechanism based on virtual population variation, experiments have found that a small CR value is better, so CR is taken as 0.1. The bandwidth of mean shift clustering is set to 0.01, and the σ of elite learning is ini and σ termin They are set to 1.0E-7 and 1.0E-10 respectively. For the DEGL algorithm, its weight factor w is selected by randomly assigning values ​​in the range of (0,1) to introduce richer diversity. For the VNCDE algorithm, the standard deviation σ required for the mutation of individual "dominants" is set to 1.0E-7.

[0142] Next, three experiments were designed around interior point search performance, transformation matrix accuracy, and iteration performance, aiming to demonstrate the overall performance of the proposed method and verify the advanced nature of the proposed method.

[0143] In order to evaluate the improvement of the FDESAC algorithm compared with the DESAC algorithm, a comprehensive comparative experiment will be conducted between them. Considering that when the number of inliers in the population is consistent, the algorithm continues to run without significant improvement, and the iteration stop condition is set to the consistent number of inliers.

[0144] For the performance of interior point search, six indicators are set: average number of interior points (average interior point rate), maximum number of interior points (maximum interior point rate), number of hypothesis models, number of verifications of a single hypothesis model, number of iterations, and running time. Among them, the average number of interior points (average interior point rate) reflects the expected value of the interior point search capability, the maximum number of interior points (maximum interior point rate) reflects the limit of the interior point search capability, and the running time reflects the time performance of the algorithm. They are the main evaluation indicators.

[0145] For the transformation matrix accuracy, two evaluation indicators are set: RMSE reprojection error and average inlier error calculated from the real matching point pairs provided in the dataset. pFor the image registration, RMSE reprojection error can more effectively evaluate the accuracy of image registration. A smaller average inlier error means that the inliers in the two images are more accurately associated, which can be used to evaluate the accuracy of the transformation matrix from the side. The calculation formula of RMSE reprojection error is as shown below, where (p, p′) is the real matching point pair provided by the dataset, k is the number of real matching point pairs, and f is the transformation model solved by the algorithm.

[0146]

[0147] Table 1 shows the experimental results of the two methods on the inlier search performance, where the values ​​with black text on a gray background are the optimal values. In general, the inlier search performance of the FDESAC algorithm is not as good as that of the DESAC algorithm, but the difference is small. In the image pairs (11) and (13), the average number of inliers of the FDESAC algorithm even exceeds that of the DESAC algorithm. For the maximum number of inliers, the FDESAC algorithm is consistent with the DESAC algorithm. However, the time performance of the FDESAC algorithm is very good. Figure 5 The speedup ratios of different image pairs are given, which are defined as the ratio of the running time of the DESAC algorithm to the FDESAC algorithm. It can be seen that the speedup ratios of most images exceed 2.5, the highest and lowest speedup ratios are 4.62 and 1.69 respectively, and the average speedup ratio reaches 3.14. From Table 1, we can see that this is because the three strategies of the FDESAC algorithm can effectively reduce the number of hypothesis models, the number of verifications of a single hypothesis model, and the number of iterations.

[0148] Table 2 shows the experimental results of the two algorithms on the accuracy of the perspective matrix. For the RMSE reprojection error and the average inlier error, the two algorithms have similar means in all images, and the FDESAC algorithm is slightly lower. The number of image pairs where the FDESAC algorithm outperforms the DESAC algorithm is 8 and 11, respectively. Therefore, FDESAC can effectively reduce the running time while maintaining or even slightly improving the accuracy of the perspective matrix.

[0149] Table 1 Experimental results of DESAC algorithm and FDESAC algorithm on interior point search performance

[0150]

[0151]

[0152] Table 2 Experimental results of DESAC algorithm and FDESAC algorithm on perspective matrix accuracy

[0153]

[0154] Next, three experiments were designed around interior point search performance, transformation matrix accuracy, and iteration performance, aiming to demonstrate the overall performance of the FDESAC algorithm and verify the advanced nature of the proposed method.

[0155] Inlier search performance: The FDESAC algorithm will be comprehensively compared with advanced RANSAC variants. First, all methods will be run under equal total validation times, comparing their total average inlier number and total maximum inlier number in all image pairs. The iteration stop condition of all algorithms is set to the same total validation times. When the total validation times of other algorithms are consistent with that of the FDESAC algorithm, the algorithm terminates.

[0156] Figure 6 The total average number of inliers and the total maximum number of inliers for all image pairs of different algorithms are given. It can be seen that the total average number of inliers and the total maximum number of inliers of the FDESAC algorithm are the highest, indicating that it has the strongest inlier search capability with relatively few verification times. The two indicators of the benchmark RANSAC algorithm are both the lowest and the difference is large. The difference between it and other algorithms in the maximum number of inliers is smaller than the difference in the average number of inliers. The trends of the two indicators of the CSDE, VNCDE, DESAC, and FDESAC algorithms are almost consistent. The PSOSAC algorithm is inferior to the PERANSAC algorithm in the average number of inliers, but better than the PERANSAC algorithm in the maximum number of inliers. This indirectly shows that RANSAC based on parameter space optimization has more advantages in searching for the maximum number of inliers.

[0157] Transformation matrix accuracy: The FDESAC algorithm is compared with other algorithms in terms of transformation matrix accuracy. All methods are still run under the same total number of validations, and the total RMSE reprojection error and the total average inlier error of all images are compared. Figure 7 , Figure 8The total RMSE reprojection error and total average inlier error of different algorithms in all image pairs are given respectively. For the total RMSE reprojection error, the FDESAC algorithm has the smallest value among all algorithms and is lower than the DESAC algorithm. The four RANSAC algorithms improved based on the differential evolution algorithm perform well because they can all return a consistent set with a larger number of inliers. The larger the number of inliers, the more reliable the consistent set is. However, since the other algorithms perform poorly in image pairs (8) and image pairs (11), the smaller number of inliers leads to unreliable transformation models corresponding to the consistent set, which produces a higher RMSE reprojection error. For the total average inlier error, the FDESAC algorithm is lower than the PSOSAC, CSDE, VNCDE, and DESAC algorithms, and higher than the RANSAC, PERANSAC, and AGASAC algorithms. This is because these three algorithms only return a small number of inliers in most runs, and the errors of these inliers are relatively small, while the transformation models corresponding to a small number of inliers are often rough. FDESAC can control the average inlier error to a low level while finding as many inliers as possible.

[0158] Iterative performance: In order to observe the changing trend of the number of internal points of different algorithms when the number of verifications gradually increases, an iterative performance experiment with a fixed number of verifications was set as the iteration stop condition. Observing the changes in the number of internal points can intuitively reflect the operating efficiency of the algorithm. Fig. 9 The average and maximum inliers of different algorithms are given when the total number of verifications is from 5*NP*|S| to 200*NP*|S| (interval is 5*NP*|S|). The experiment is conducted on image pairs (8) with extremely low inlier rate (~2%) and extremely high total number of samples (9289).

[0159] For the average number of inliers, the FDESAC algorithm grows slower than the DESAC algorithm when it is within 40*NP*|S|, but it is higher than the DESAC algorithm when it is between 40*NP*|S| and 150*NP*|S|. When it is between 150*NP*|S| and 200*NP*|S|, both the FDESAC algorithm and the DESAC algorithm tend to converge to a similar average number of inliers. The curves of the CSDE algorithm and the VNCDE algorithm are similar. In the early and middle stages, the CSDE algorithm is slightly better than the VNCDE algorithm, while in the later stages, the VNCDE algorithm is better than the CSDE algorithm. The remaining algorithms perform poorly, among which the benchmark RANSAC algorithm has the worst performance, and it quickly and stably converges to an extremely low value.

[0160] As for the maximum number of inliers, the FDESAC algorithm is better than the DESAC algorithm throughout the process. It only needs to converge to the maximum number of inliers from 5*NP*|S| to 25*NP*|S|. The DESAC algorithm needs to converge from 5*NP*|S| to 110*NP*|S| to achieve this. The VNCDE algorithm is better than the CSDE algorithm in searching for the maximum number of inliers. Although the other algorithms occasionally search for a higher maximum number of inliers, they are extremely unstable. The benchmark RANSAC algorithm still has the worst performance in searching for the maximum number of inliers. Therefore, the FDESAC algorithm has excellent time performance. It can search for a higher number of inliers with fewer verifications and return a more reliable consistent set and transformation model.

[0161] In order to study the improvement of the FDESAC algorithm by various improvements, the experimental results of removing various improvements from the FDESAC algorithm are given, and then the influence of different scaling factors F on the FDESAC algorithm is analyzed. Table 3 shows the experimental results of removing various improvements from the FDESAC algorithm, and the iteration stop condition is that the number of points in the population of the algorithm is consistent. Among them, algorithms Remove1, Remove2, and Remove3 respectively represent the FDESAC algorithm with fast inspection removed, improved resident initialization, and F set to a constant value of 0.75. The values ​​with black text on a gray background are the optimal values.

[0162] In general, algorithm Remove3 has the best comprehensive average inliers and maximum inliers due to its high scaling factor F. The FDESAC algorithm with three improvements has the best time performance in most image pairs. Among them, fast verification is the biggest improvement in speed. After removing the fast verification, the number of hypothesis models and iterations of algorithm Remove1 do not change much, but the number of verifications for each model will increase sharply, which will directly increase the running time of the algorithm. For algorithm Remove2 with improved resident initialization removed, the quality of the generated model deteriorates, the number of hypothesis models and iterations required to be generated will increase, and the number of verifications for each model will also decrease accordingly. Therefore, the average number of inliers of algorithm Remove2 is the lowest among the four methods. Although the running time is similar to that of the FDESAC algorithm, it is still not as good as that of the FDESAC algorithm. For algorithm Remove3, a high F value will directly improve the accuracy of the algorithm, but it will also increase the number of hypothesis models and iterations, which will increase the running time of the algorithm.

[0163] Table 3 Experimental results of removing various improvements

[0164]

[0165]

[0166]

[0167] In order to more directly show the accuracy and speed comparison of the FDESAC algorithm using different F values ​​and the adaptive F strategy, Fig.10 The average number of inliers and running time of the FDESAC algorithm with different F values ​​on the image pair (8) are given. The green horizontal line represents the average number of inliers of the FDESAC algorithm using the adaptive F strategy, and the blue horizontal line represents its running time. The black line is used here to connect the two points corresponding to the two values ​​of the FDESAC algorithm using the adaptive F strategy. The smaller the slope of the black line, the more effective the strategy is in searching for a higher number of inliers in a shorter time, and the strategy is more effective than simply adjusting the F value. It can be seen that the average number of inliers and running time increase accordingly with the increase of F. And the black line is not vertical, but tilted from the lower left to the upper right. Therefore, the FDESAC algorithm using the adaptive F strategy is more effective than simply adjusting the F value. Overall, the three improvements proposed in this paper are effective, and they together improve the running efficiency of the DESAC algorithm.

[0168] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image registration method based on the historical record fast differential evolution sampling consensus algorithm, characterized in that: The following steps are involved: Step 1: Image acquisition: The image acquisition module acquires an image to be registered I1(x, y) and a reference image I2(x′, y′), and passes them to the feature extraction module; Step 2: Feature extraction: The feature extraction module uses the SIFT algorithm to extract the point features of the image to be registered I1 (x, y) and the reference image I2 (x′, y′), respectively, to obtain the feature set F1 of the point to be registered and the feature set F2 of the reference point, and pass them to the feature matching module; Step 3: Feature matching: The feature matching module performs rough feature matching on the feature set F1 of the points to be registered and the feature set F2 of the reference points to obtain a set of matching point pairs, i.e., a sample set S, and passes the sample set S to the image registration module of the fast differential evolution sampling consensus algorithm FDESAC based on historical records; Step 4: Parameter setting: The image registration module sets parameters according to the sample set S: SIFT feature distance ratio D r , error threshold t for determining internal points, population size NP, weight factor mean μw, pre-test sampling number N f Set up and set the neighborhood range Nr = NP / 10, the initial number of internal points n f =N f ; Step 5: The image registration module registers the sample set S according to the SIFT feature distance ratio D r Sort in ascending order, then calculate BP based on the sample set S, and initialize the sample internal point file A S and model interior point file A M ; According to the pre-test sampling number N f and the initial number of interior points n f Calculate the fast test execution threshold uT; Step 6: Initialize the population for the first time: Randomly select m samples from the first BP*|S| range of the sample set S to initialize the individual X i , through the test function Test for individual X i , i∈[0,NP] to test, and the test results are i Evaluation index EI i , and update the model internal point file A M ; Step 7: According to the pre-test sampling number N f And the number of inner points Inlier(X) updates the fast test execution threshold uT; Step 8: Mutation phase: Initialize the weight coefficient set S of the mutant individuals that successfully enter the next round of iteration w , according to the weight coefficient set S w Calculate the scaling factor F i , according to the scaling factor F i Calculate the variant individual V i ; Step 9: Selection phase: Use the test function Test to select the variant individual V i Test and get the variant individual V i Evaluation index EI y ; Individual X i Evaluation index EI i With the variant individual V i Evaluation index EI y For comparison, if the variant individual V i Evaluation index EI y Better than individual X i Evaluation index EI i , then the individual X i Replaced with variant individual V i , the mutant individual V i The weight factor w is recorded in the weight coefficient set S w middle; Otherwise, randomly select m samples from the first BP*|S| range of the sample set S to reinitialize individual X i , through the test function Test for individual X i Test and get individual X i Evaluation index EI i , and update the model internal point file A M ; Step 10: Update mutation operator parameters: Calculate weight coefficient set S w The mean avg of w The mean avg of the weight factor is calculated as the mean μw; Step 11: Resident initialization population: Calculate the number of resident initialization repetitions r, if the mutation is successful, the number of individuals | S w | is equal to 0, then r = 3*NP; otherwise, r = 3*NP / |S w |; Find the worst individual X in the population worst ; If iter>N f And iter%uT≠0, iter represents the number of iterations, randomly select m model internal point records a in the first BP*|S| range of the sample set S m ≠0 samples reinitialize individual X new ; Otherwise, randomly select m samples from the first BP*|S| range of the sample set S to reinitialize individual X new ; Through the test function Test, the individual X new Test and get individual X new Evaluation index EI new , and update the model internal point file A M ; Individual X new Evaluation index EI new and the worst individual X worst Evaluation index EI worst For comparison, if individual X new Evaluation index EI new Better than the worst individual X worst Evaluation index EI worst , then the worst individual X worst Replace with individual X new ; Step 12: Repeat steps 7-11 until the preset number of iterations is reached, then proceed to step 13; Step 13: Select the best individual in the population: Select the individual with the best evaluation index EI from the population and save it to the best individual X Best , the optimal individual X Best As a spatial geometric transformation model, the image to be registered I1 (x, y) is spatially transformed to obtain a registered image.

2. The image registration method based on the historical record fast differential evolution sampling consensus algorithm according to claim 1 is characterized by: In step 2, the SIFT algorithm extracts point features in various scales of space, including but not limited to corner points, edge points, bright spots in dark areas, and dark spots in bright areas.

3. The image registration method based on the historical record fast differential evolution sampling consensus algorithm according to claim 1 is characterized by: In step 5, the expression of BP is calculated according to the sample set S as follows: Among them, m is the minimum number of samples to generate the model, |S| is the total number of samples; The expression for calculating the fast test execution threshold uT is as follows:

4. The image registration method based on the historical record fast differential evolution sampling consensus algorithm according to claim 1 is characterized by: In step 8, the scaling factor F is calculated i The expression is as follows: Among them, rand(0.6,0.75) is a uniformly distributed random number with a value between (0.6,0.75), |S w | is the number of successfully mutated individuals in the previous generation of the population, NP is the population size, and g represents the gth generation; Calculate the variant individual V i The expression is as follows: V i =wV rand1 +(1-w)V local_best Among them, the i-th variant individual V i By randomly mutating individual V rand1 and the optimal mutation individual V in the neighborhood local_best The weighted value is obtained by weighting factor w, w∈[0,1]; F is the scaling factor, X local_best Is individual X i The best neighbor within the index range of .

5. The image registration method based on the historical record fast differential evolution sampling consensus algorithm according to claim 1, characterized in that: In step 10, the expression for calculating the mean value μw of the weight factor is as follows: μw=(1-0.1)*μw+0.1*avg.

6. The image registration method based on the historical record fast differential evolution sampling consensus algorithm according to claim 1, characterized in that: In step 12, the entire iterative process is divided into an archive recording phase and an archive application phase, and the dividing point between the two phases is the maximum sample inlier record max(a s )=NP; For the archival recording stage, the test function Test is used to test the individual X i The inspection method is pre-inspection + full inspection; In the archive application stage, the test function Test is used to test the individual X i The inspection method is pre-inspection + rapid inspection; When performing a quick test, if sample s i The sample point record a s,i If it is 0, then the verification is skipped and it is directly judged as an outlier; In the archive application stage, when the population iteration number iter%uT=0, the pre-check + quick check is not performed, but the pre-check + full check is performed instead.

7. The image registration method based on the historical record fast differential evolution sampling consensus algorithm according to claim 1 is characterized by: The model internal point file A M Updated when samples are drawn to generate the hypothesis model, i.e., when the population is initialized; First, the algorithm randomly selects m samples from the excellent samples in the range of the first BP*100% to construct the minimum sample subset S min ={s x1 ,s x2 ,…,s xm }, where m is the minimum number of samples for building the model; then, using S min Generate a hypothesis model X and verify it to obtain the number of internal points Inlier(X); finally, for each sample s involved in building the hypothesis model X xi , the corresponding model internal point record a m,xi The update expression is as follows: a m,xi =a m,xi +Inlier(X)。 8. The image registration method based on the historical record fast differential evolution sampling consensus algorithm according to claim 1 is characterized by: In step 4, the SIFT feature distance ratio D r is set to 0.8, and the error threshold t for determining the internal point is set to 5 pixels.

9. The image registration method based on the historical record fast differential evolution sampling consensus algorithm according to claim 1, characterized in that: The evaluation index EI includes two indicators: the number of inliers and the inlier error.

Images