Image registration method based on Sober edge detection

Abstract

The invention discloses an image registration method based on sober edge detection. The method comprises the following steps: acquiring a polarization reference image A and a polarization to-be-registered image B; performing feature extraction on the polarized reference image A and the polarized to-be-registered image B by using a sober edge detection algorithm; performing maximum similarity point location matching on the extracted features of the polarization reference image A and the extracted features of the polarization image B to be registered by using an improved normalized cross-correlation matching method; a random sampling consistency method is used for eliminating wrong matching pairs of the point locations with the maximum similarity, a translation matrix is constructed through the corresponding relation of the point locations with the maximum similarity, and construction of a translation parameter model M is completed; and performing translation transformation and image superposition on the polarized image B to be registered according to the translation parameter model M to obtain a registration result. Compared with a traditional registration method based on gray scale and characteristics, the method provided by the invention is simpler and more convenient to implement, higher in applicability and higher in registration precision.

Description

Technical Field

[0001] The invention belongs to the technical field of image registration, and relates to an image registration method based on Sober edge detection, in particular to a new polarization image registration method. Background Art

[0002] Infrared polarization imaging technology is an advanced technology that uses infrared radiation and polarization characteristics to acquire and analyze images. It combines infrared imaging technology with polarization optical principles, and mainly uses the differences in infrared radiation polarization characteristics between artificial targets and natural targets, real targets and camouflaged targets to improve target recognition. It can improve information about the texture, material, surface roughness, etc. of the ground objects, thereby raising target recognition from surface structure to spatial structure, providing multiple options for describing targets in infrared scenes, and is more conducive to detecting infrared radiation information and polarization information of target objects. It is widely used in materials science, biomedicine, military reconnaissance and other fields.

[0003] Image sub-pixel registration technology has gone through the development from integer pixel level to sub-pixel interpolation, and then to feature point sub-pixel matching and optimization-based sub-pixel registration methods. It can be roughly divided into region-based and feature-based. At present, the region-based image sub-pixel registration methods mainly include interpolation method and optimization method. Among them, the interpolation method uses interpolation technology to interpolate and resample the image and template, and performs pixel-level registration on the sampled image to obtain sub-pixel accuracy. The performance of registration accuracy is affected by the interpolation method; although the optimization method has high solution accuracy, the amount of calculation is very large in practical applications, the calculation speed is slow, and the convergence probability and the probability of finding the optimal solution need to be improved. With the introduction of image pyramid and deep learning technology, the accuracy and robustness of sub-pixel registration have been further improved. The feature-based image registration method mainly extracts the same feature points of the reference image and the image to be registered, then uses matching to form the corresponding transformation relationship, and finally obtains the registered image through the transformation relationship. The development of these technologies has made image registration widely used in medical imaging, remote sensing images, computer vision and other fields.

[0004] However, these methods are all registration methods for traditional images. Polarized images lack obvious feature information. When using the image registration method of non-polarized images (ordinary images) for polarized image registration, there are problems such as difficulty in feature extraction, poor registration effect, small application range, and complex registration steps. Therefore, how to develop a simple, easy and widely used polarized image registration method has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to solve the problems of difficult feature extraction, poor registration effect, small application scope and complicated registration steps when using the image registration method of non-polarized image (ordinary image) to perform polarized image registration, the technical solution adopted by the present invention is: an image registration method based on Sober edge detection, comprising the following steps:

[0006] S1: Acquire polarization image: acquire polarization reference image A and polarization image to be registered B;

[0007] S2: Feature extraction of polarization images: Use the Sober edge detection algorithm to extract features from the polarization reference image A and the polarization image to be registered B respectively;

[0008] S3: Feature point matching: Use the improved normalized cross-correlation matching method to perform maximum similarity point matching on the features extracted from the polarization reference image A and the features extracted from the polarization image to be registered B;

[0009] S4: Obtaining the translation parameter model: using the random sampling consistency method to eliminate the wrong matching pairs after the maximum similarity point matching, constructing the translation matrix through the correspondence relationship of the maximum similarity points, and completing the construction of the translation parameter model M;

[0010] S5: Obtaining the registration result: performing translation transformation and image superposition on the polarized image to be registered B according to the translation parameter model M to obtain the registration result.

[0011] Furthermore: the method is applicable when the sizes of the polarization reference image A and the polarization image to be registered B are the same.

[0012] Furthermore: the feature extraction of the polarization image also includes preprocessing before feature extraction:

[0013] The polarization reference image A and the polarization image to be registered B are converted from color to grayscale images so that the result only contains the amplitude of the pixel and an image containing edge intensity information is obtained; for the reference image A, when it has a different image resolution from the polarization image B, image A needs to be scaled, and by defining a scaling factor, the reference image A is scaled to have the same number of pixels and image size as the reference image B.

[0014] Further: the process of extracting features from the polarized reference image A and the polarized image to be registered B by the Sober edge detection algorithm is the same. The process of extracting features from the polarized reference image A by the Sober edge detection algorithm is as follows:

[0015] The Sober edge detection algorithm is used to calculate the gradient information of the pixel point A of the polarization reference image to perform edge detection operations and extract the feature information of the polarization image. The convolution formula of the Sober matrix is ​​as follows:

[0016] g=f*h

[0017]

[0018] Where: g = f*h represents the convolution operation on the image;

[0019] h: represents the convolution kernel used in the Sober edge detection algorithm, which is used to calculate the gradient information of each pixel in the image. It extracts edge features by performing a convolution operation with the local area of ​​the image f;

[0020] f: represents the original image, that is, the reference polarization image A or the polarization image B to be registered.

[0021] f(ik,jl): represents the result of multiplying the pixel value of the original image f at point f(ik,jl) by the weight value h(k,l) of the convolution kernel h at point (k,l). By multiplying each weight h(k,l) in the convolution kernel with the pixel value f(ik,jl) at the corresponding position in the image f. Among them, h(k,l) is the relative position coordinate in the convolution kernel, and (ik,jl) represents the absolute position coordinate in the image, which is determined by aligning the center of the convolution kernel with the pixel point in the image.

[0022] ∑ k,l : represents the sum of all k and l values ​​in the convolution kernel h. For each k and l, f(ik,jl) is the value of the pixel in image f aligned with the center of the convolution kernel, and h(k,l) is the weight of the convolution kernel at the (k,l) position.

[0023] i, j: the horizontal and vertical coordinates of the pixel point in the area with the maximum gradient value threshold in the polarization reference image A or the polarization image to be registered B, respectively;

[0024] k, l: represent the horizontal and vertical coordinates of the position of the element in the convolution kernel h. During the convolution operation, k and l will traverse all elements of the convolution kernel and multiply each element of the convolution kernel with the pixel value of the corresponding position in the local area of ​​the image f, and finally sum them up to obtain the gradient information of the local area;

[0025] The Sober convolution factor contains two sets of 3x3 matrices, one for horizontal and one for vertical. The two matrices are convolved with the polarization reference image A to obtain the approximate brightness difference in the horizontal and vertical directions. The polarization reference image A represents the original image, and Gx and Gy represent the gradient amplitudes detected by the horizontal and vertical edges, respectively. The formula is as follows:

[0026]

[0027]

[0028] The polarization reference image A data is multiplied and added with the operator value at the corresponding position to obtain the horizontal gradient Gx and vertical gradient Gy of each pixel in the image. Then the gradient amplitude (G) and direction (θ) of each pixel are calculated. The specific formula is as follows:

[0029]

[0030] θ=arctan(Gy / Gx)+(1 / 4)π

[0031] Compare the calculated gradient amplitude (G) with the set threshold. If the gradient amplitude (G) is greater than the threshold, the point (f, h) is considered to be an edge point and displayed as a black point. Otherwise, it is displayed as a white point. The gradient amplitude is binarized to obtain the final edge image.

[0032] When the amplitude G is less than or equal to the set threshold, it indicates that the edge strength of the pixel is weak.

[0033] Further: the process of performing maximum similarity point matching on the features extracted from the polarization reference image A and the features extracted from the polarization image to be registered B using the improved normalized cross-correlation matching method is as follows:

[0034] The improved normalized cross-correlation matching method is used to find the pixel position with the maximum similarity. The calculation formula is as follows:

[0035]

[0036] The formula is used to measure the correlation or similarity between the polarization reference image and the polarization image to be registered. The numerator represents the sum of the products of the pixel values ​​of the polarization image and the polarization image to be registered, and the denominator represents the square root of the sum of the squares of the pixel values ​​of the polarization reference image and the polarization image to be registered.

[0037] Where: value (R) represents the similarity between two pixel points, f(i,j) and g(i,j) represent the pixel values ​​in the polarization reference image, where (i,j) are the two-dimensional coordinates of the pixel in the polarization image to be registered, i represents the row index, ranging from 1 to M; j represents the column index, ranging from 1 to N;

[0038] M represents the height of the polarization reference image, that is, the number of rows of pixels in the polarization reference image;

[0039] N represents the width of the polarization reference image, that is, the number of columns of pixels in the polarization reference image;

[0040] H represents the height of the polarization image to be registered, that is, the number of rows of pixels in the polarization image to be registered;

[0041] W represents the width of the polarization image to be registered, that is, the number of columns of pixels in the polarization image to be registered;

[0042] The matching is completed according to the bidirectional matching strategy. The bidirectional matching strategy is that the two characteristic pixel points in the polarization reference image A and the polarization image to be registered B must satisfy the maximum correlation coefficient with each other to become a matching pair.

[0043] Further: the process of constructing the translation matrix through the correspondence relationship of the maximum similarity points and completing the construction of the translation parameter model M is as follows:

[0044] 41): Assume that a certain pixel value similarity point set is C = {c1, c2, ... cs}, and randomly sample a pair of matching pairs S;

[0045] 42) After removing a pair of randomly selected matching points from set C, calculate the root mean square error between the remaining feature matching point pairs (C') in set C after removing set S and set S; then repeat the above steps until the root mean square error between each matching point pair set S and the remaining feature matching point pairs (C') is obtained. If the root mean square error is less than the set threshold t, it is considered to be the optimal matching point pair for the pixel;

[0046] 43) The steps for solving the root mean square error (RMSE) are as follows: assume that the maximum similarity value in the matching pair set S is the actual value, and the remaining feature matching point pairs (C') are the predicted values; then calculate the error value between the predicted value and the actual value, the sum of squares of the error values, the mean square error (MSE), and the root mean square error (RMSE) in turn. If the root mean square error is lower than the threshold t, the matching point pair is considered to be the optimal matching point pair for the pixel point, which is S = {sm}, and the values ​​corresponding to the two pixel value similarity points in the matching point are f(i, j) and g(i, j).

[0047] 44) Repeat steps 41)-43) until all the optimal matching point pairs are found. At this time, the matching pair set is S = {s1, s2, ... sm}, and it is used as the final translation parameter model M. Among them, the values ​​corresponding to the two pixel value similarity points in the i-th pair of matching points are f(i, j) and g(i, j).

[0048] Furthermore: the acquisition of the registration result specifically includes the following steps:

[0049] 51) According to the position of the maximum pixel similarity point between the polarization reference image A and the polarization image to be registered B, the sub-pixel translation (tx i,ty i ), and then find the translation amount between the polarization reference image A and the polarization image to be registered B;

[0050] 52) constructing a translation matrix from the translation parameter model M;

[0051] 53) The polarization reference image A and the polarization image to be registered B are translated and superimposed to complete the matching and obtain the matching result.

[0052] The present invention provides an image registration method based on sober edge detection, which has the following advantages: compared with the prior art, a new edge detection method is proposed, which makes polarization image registration simpler. The present invention includes the following steps: acquiring polarization images; extracting features using sober edge detection; matching feature points; obtaining translation parameter models; and obtaining registration results. The present invention is based on the special image characteristics of polarization images and realizes accurate registration between polarization images. Compared with the prior art, the present invention is simpler to implement and can perform higher-precision registration of polarization images in various application ranges, and can meet sub-pixel requirements to meet the increasing demand for high-precision work.

[0053] Various application areas include but are not limited to the following:

[0054] First, in materials science and engineering, polarization image registration can be used to study the microstructure and organization of materials. By registering polarization images at different angles or under different loading conditions, the performance and characteristics of the material can be analyzed more accurately.

[0055] Second, polarization image registration can be used for the alignment and registration of medical images, such as corneal imaging and histological research in the field of ophthalmology. Through polarization image registration, the accuracy and visualization of medical images can be improved, helping doctors make more accurate diagnosis and treatment decisions.

[0056] Third, during the semiconductor manufacturing process, polarization image registration can be used to precisely align masks and silicon wafers to ensure accurate pattern transfer, detect, and analyze microscopic defects in semiconductor devices.

[0057] Fourth, polarization image registration can be used to monitor the health of plants, the growth of farmland, and the maturity of crops. By registering polarization images at different time points, we can better understand the changes and trends in plant growth.

[0058] Aiming at the characteristics of lack of specific feature information in polarization images and large difference in grayscale distribution between images, the present invention selects the sober edge detection operator with image grayscale change and noise reduction to extract feature information from polarization images; then uses the improved normalized cross-correlation matching method to find the maximum similarity point; then removes the wrong matching pairs according to the random sampling consistency method, constructs the translation matrix through the correspondence relationship of the maximum similarity point, and completes the construction of the translation parameter model M. Finally, the polarization image B to be registered is translated according to the translation parameter model M and superimposed with the image to obtain the registration result.

[0059] The present invention combines the advantages of feature information based on polarization images and the sober edge detection algorithm, purifies matching pairs through the geometric relationship of feature matching points and the use of the (RANSAC) algorithm, thereby obtaining the overall transformation relationship of the image, and then constructs a translation parameter model M through the correspondence relationship of the maximum similarity points for image registration. Compared with the traditional registration method based on grayscale and features, the present invention is simpler to implement, more applicable, and has higher registration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0061] Figure 1 is a flow chart of the method of the present invention;

[0062] Figure 2 (a) Using the image B with a polarization angle of 0 degrees as the reference image, the image obtained by the method of the present invention at a polarization angle of 60 degrees and the difference image of the reference image are obtained; (b) Using the image B with a polarization angle of 0 degrees as the reference image, the image obtained by the traditional method at a polarization angle of 60 degrees and the difference image of the reference image are obtained; (c) Using the image B with a polarization angle of 0 degrees as the reference image, the image obtained by the method of the present invention at a polarization angle of 120 degrees and the difference image of the reference image are obtained; (d) Using the image B with a polarization angle of 0 degrees as the reference image, the image obtained by the traditional method at a polarization angle of 120 degrees and the difference image of the reference image are obtained; Figure 3 (a) is the polarization reference image A, (b) is the polarization image to be registered B, and (c) is the image after registration and superposition. DETAILED DESCRIPTION

[0063] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0064] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] An image registration method based on sober edge detection comprises the following steps:

[0066] S1: Acquire polarization image: acquire polarization reference image A and polarization image to be registered B;

[0067] S2: Feature extraction of polarization images: Use the sober edge detection algorithm to extract features from the polarization reference image A and the polarization image to be registered B respectively;

[0068] S3: Feature point matching: Use the improved normalized cross-correlation matching method to perform maximum similarity point matching on the features extracted from the polarization reference image A and the features extracted from the polarization image to be registered B;

[0069] S4: Obtaining the translation parameter model: using the random sampling consistency method to eliminate the wrong matching pairs after the maximum similarity point matching, constructing the translation matrix through the correspondence relationship of the maximum similarity points, and completing the construction of the translation parameter model M;

[0070] S5: Obtaining the registration result: performing translation transformation and image superposition on the polarized image to be registered B according to the translation parameter model M to obtain the registration result.

[0071] Steps S1 / S2 / S3 / S4 / S5 are executed sequentially;

[0072] This method is applicable when the size of the polarization reference image A and the polarization image to be registered B is the same. Before starting this method, the two polarization images are preprocessed to check the shape information of the two polarization images, including length, width and other information;

[0073] The feature extraction of the polarization image also includes preprocessing before feature extraction:

[0074] The polarization reference image A and the polarization image to be registered B are converted from color to grayscale images so that the result contains only the pixel amplitude and an image containing edge intensity information is obtained; for the reference image A, when it has a different image resolution from the polarization image B, image A needs to be scaled, and by defining a scaling factor, the reference image A is scaled to have the same number of pixels and image size as the reference image B.

[0075] According to the characteristics of the lack of feature information that can be used for identification of targets in polarized images and the large difference in grayscale distribution between images. At the same time, the image constructed by the Sober edge detection operator has very high stability in grayscale changes, noise influence and viewing angle changes, and the operation is simple, and the amount of feature information extracted is also large. Therefore, the Sober edge detection algorithm is used to obtain the coordinates of the feature points.

[0076] The Sober edge detection algorithm is used to calculate the gradient information of pixels in the polarization image to perform edge detection operations and extract the feature information of the polarization image.

[0077] The process of extracting features from the polarized reference image A and the polarized image to be registered B using the Sober edge detection algorithm is the same. The process of extracting features from the polarized reference image A using the Sober edge detection algorithm is as follows:

[0078] The Sober edge detection algorithm is used to calculate the gradient information of the pixel point A of the polarization reference image to perform edge detection operations and extract the feature information of the polarization image. The convolution formula of the Sober matrix is ​​as follows:

[0079] g=f*h

[0080]

[0081] Where: g = f*h represents the convolution operation on the image;

[0082] h: represents the convolution kernel used in the Sober edge detection algorithm, which is used to calculate the gradient information of each pixel in the image. It extracts edge features by performing a convolution operation with the local area of ​​the image f;

[0083] f: represents the original image, that is, the reference polarization image A or the polarization image B to be registered.

[0084] f(ik,jl): represents the result of multiplying the pixel value of the original image f at point f(ik,jl) by the weight value h(k,l) of the convolution kernel h at point (k,l). By multiplying each weight h(k,l) in the convolution kernel with the pixel value f(ik,jl) at the corresponding position in the image f. Among them, h(k,l) is the relative position coordinate in the convolution kernel, and (ik,jl) represents the absolute position coordinate in the image, which is determined by aligning the center of the convolution kernel with the pixel point in the image.

[0085] ∑ k,l : represents the sum of all k and l values ​​in the convolution kernel h. For each k and l, f(ik,jl) is the value of the pixel in image f aligned with the center of the convolution kernel, and h(k,l) is the weight of the convolution kernel at the (k,l) position.

[0086] i, j: the horizontal and vertical coordinates of the pixel point in the area with the maximum gradient value threshold in the polarization reference image A or the polarization image to be registered B, respectively;

[0087] k, l: represent the horizontal and vertical coordinates of the position of the element in the convolution kernel h. During the convolution operation, k and l will traverse all elements of the convolution kernel and multiply each element of the convolution kernel with the pixel value of the corresponding position in the local area of ​​the image f, and finally sum them up to obtain the gradient information of the local area;

[0088] Let Sober convolution factor:

[0089] -1 0 +1 -2 0 +2 -1 0 +1

[0090] Gx:

[0091] +1 +2 +1 0 0 0 -1 -2 -1

[0092] Gy:

[0093] The Sober convolution factor includes two sets of 3x3 matrices, one for horizontal and one for vertical, which are convolved with the polarization reference image A to obtain approximate values ​​of horizontal and vertical brightness differences.

[0094] The polarization reference image A represents the original image, Gx and Gy represent the gradient amplitudes detected by horizontal and vertical edges respectively, and the formulas are as follows:

[0095]

[0096] The polarization reference image A data is multiplied and added with the value of the operator at the corresponding position to obtain the horizontal gradient Gx and vertical gradient Gy of each pixel in the image. Then the gradient amplitude (G) and direction of each pixel are calculated. The specific formula is as follows:

[0097]

[0098] θ=arctan(Gy / Gx)+(1 / 4)π

[0099] The calculated gradient amplitude (G) is compared with the set threshold. If (G) is greater than the threshold, the point (f, h) is considered to be an edge point and displayed as a black point, otherwise it is displayed as a white point. The gradient amplitude is then binarized to obtain the final edge image.

[0100] When the calculated gradient magnitude (G) is less than or equal to the set threshold, it means that the edge strength of the pixel is weak and it is not the significant edge pixel we are looking for. We can ignore the pixel or combine it with other features of the image to perform edge connection or tracking.

[0101] Feature point matching uses the improved normalized cross-correlation matching method (R value formula) to calculate the cross-correlation value between feature points, completes the matching according to the two-way matching strategy, and achieves the maximum similarity point matching of the primary selected features. Compared with the traditional matching method, the matching accuracy is greatly improved.

[0102] The process of performing maximum similarity point matching on the features extracted from the polarization reference image A and the features extracted from the polarization image to be registered B using the improved normalized cross-correlation matching method is as follows:

[0103] The improved normalized cross-correlation matching method is used to find the pixel position with the maximum similarity. The calculation formula is as follows:

[0104]

[0105] The formula is used to measure the correlation or similarity between the polarization reference image and the polarization image to be registered. The numerator represents the sum of the products of the pixel values ​​of the polarization image and the polarization image to be registered, and the denominator represents the square root of the sum of the squares of the pixel values ​​of the polarization reference image and the polarization image to be registered.

[0106] Where: value (R) represents the similarity between two pixel points, f(i,j) and g(i,j) represent the pixel values ​​in the polarization reference image, where (i,j) are the two-dimensional coordinates of the pixel in the polarization image to be registered, i represents the row index, ranging from 1 to M; j represents the column index, ranging from 1 to N;

[0107] M represents the height of the polarization reference image, that is, the number of rows of pixels in the polarization reference image;

[0108] N represents the width of the polarization reference image, that is, the number of columns of pixels in the polarization reference image;

[0109] H represents the height of the polarization image to be registered, that is, the number of rows of pixels in the polarization image to be registered;

[0110] W represents the width of the polarization image to be registered, that is, the number of columns of pixels in the polarization image to be registered;

[0111] The matching is completed according to the bidirectional matching strategy, and the bidirectional matching strategy is that the two feature points in the polarization reference image A and the polarization image to be registered B must satisfy the maximum correlation coefficient with each other to become a matching pair.

[0112] The bidirectional matching strategy refers to performing matching in two directions simultaneously: from the reference image to the image to be registered (forward matching), and from the image to be registered to the reference image (reverse matching).

[0113] The mutual correlation coefficient should satisfy the mutual correlation function relationship, that is, the R value function formula of the normalized mutual correlation matching method mentioned above. Only when the similarity obtained under this condition is the largest can they become a matching pair;

[0114] Furthermore, the translation parameter model is obtained by using the random sampling consensus method (RANSAC) to eliminate wrong matching pairs and obtain the best matching point information. Then, the translation parameter model "M" is constructed through the correspondence of the maximum similar points. The (RANSAC) algorithm is used to purify the matching pairs and calculate the maximum similar points. The number of feature points detected by the Sober operator is large, but they are unevenly distributed on the reference and polarization images to be registered. The model obtained by the (RANSAC) algorithm represents the global transformation parameters between images, which can reach the sub-pixel level.

[0115] The process of constructing the translation matrix through the correspondence relationship of the maximum similarity points and completing the construction of the translation parameter model M is as follows:

[0116] 41): Assume that a certain pixel value similarity point set is C = {c1, c2, ... cs}, and randomly sample a pair of matching pairs S;

[0117] 42) After removing a pair of randomly selected matching points from set C, calculate the root mean square error between the remaining feature matching point pairs (C') in set C after removing set S and set S; then repeat the above steps until the root mean square error between each matching point pair set S and the remaining feature matching point pairs (C') is obtained. If the root mean square error is less than the set threshold t, it is considered to be the optimal matching point pair for the pixel;

[0118] 43) The steps for solving the root mean square error (RMSE) are as follows: assume that the maximum similarity value in the matching pair set S is the actual value, and the remaining feature matching point pairs (C') are the predicted values; then calculate the error value between the predicted value and the actual value, the sum of squares of the error values, the mean square error (MSE), and the root mean square error (RMSE) in turn. If the root mean square error is lower than the threshold t, the matching point pair is considered to be the optimal matching point pair for the pixel point, which is S = {sm}, and the values ​​corresponding to the two pixel value similarity points in the matching point are f(i, j) and g(i, j).

[0119] 44) Repeat steps 41)-43) until all the optimal matching point pairs are found. At this time, the matching pair set is S = {s1, s2, ... sm}, and it is used as the final translation parameter model M. Among them, the values ​​corresponding to the two pixel value similarity points in the i-th pair of matching points are f(i, j) and g(i, j).

[0120] At this point, the selected set S contains a set of matching point pairs that have been verified, purified and optimized through multiple iterations. At this point, the corresponding translation parameter model M is considered to be the final translation parameter model. This translation parameter model M can be used to construct a translation transformation matrix to enable translation registration between polarization images.

[0121] Furthermore: the acquisition of the registration result specifically includes the following steps:

[0122] 51) According to the position of the maximum pixel similarity point between the polarization reference image A and the polarization image to be registered B obtained in the above steps, the sub-pixel translation amount (tx i ,ty i ), and then find the translation amount between the polarization reference image A and the polarization image to be registered B;

[0123] 52) constructing a translation matrix from the translation parameter model M;

[0124] 53) The polarization reference image A and the polarization image to be registered B are translated and superimposed to complete the matching and obtain the matching result.

[0125] By using the method proposed in the present invention, experiments were conducted using polarization images in two real scenes and compared with the traditional registration method, and polarization images with polarization angles of 0, 60 and 120 degrees were obtained in each scene. The registration accuracy was evaluated in both objective and subjective ways. The objective method was to calculate the normalized mutual information (NMI) value between the reference image and the registered image, which is a global indicator. The larger the value, the better the registration effect. The subjective method was to make a difference between the polarization reference image and the registered polarization image. The darker the image after the difference, the better the registration effect.

[0126] Based on scene 2 (polarization image B to be registered), the image with a polarization angle of 0 is used as the reference image, and the images to be registered with polarization angles of 60 and 120 are registered, and the difference between the registered image and the reference image is taken.

[0127] like Figure 2 As shown, Figure 2 (a) is the difference image between the registered image and the reference image with a polarization angle of 60 under the method of the present invention, and the calculated (NMI) value is 1.018; (b) is the difference image between the registered image and the reference image with a polarization angle of 60 under the traditional method, and the calculated (NMI) value is 1.027. (c) is the difference image between the registered image and the reference image with a polarization angle of 120 under the method of the present invention, and the calculated (NMI) value is 1.019, and (d) is the corresponding result under the traditional method, and the calculated (NMI) value is 1.028.

[0128] Figure 3 (a) represents the polarization reference image A, (b) represents the polarization image to be registered B, and (c) represents the registered image after translation transformation and image superposition.

[0129] according to Figure 2 (a)-(d) The corresponding (NMI) values ​​and Figure 3 It can be seen from the changes in images (a)-(c) that the registration effect of the present invention is significantly better than that of the traditional method for polarization image registration. The present invention is very effective for polarization image registration.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image registration method based on Sober edge detection, characterized in that: The following steps are involved: S1: Polarization image acquisition: obtain reference polarization image A and polarization image B to be registered; S2: Polarization image feature extraction: Use the Sober edge detection algorithm to extract features from the reference polarization image A and the polarization image to be registered B respectively; S3: Feature point matching: Use the improved normalized cross-correlation matching method to perform maximum similarity point matching on the features extracted from the reference polarization image A and the features extracted from the polarization image B to be registered; S4: Obtaining the translation parameter model: Using the random sampling consistency method to eliminate the wrong matching pairs after the maximum similarity point matching, construct the translation matrix through the correspondence relationship of the maximum similarity points, and complete the construction of the translation parameter model M; S5: Obtaining a registration result: performing a translation transformation and image superposition on the polarization image B to be registered according to the translation parameter model M to obtain a registration result.

2. The image registration method based on Sober edge detection according to claim 1, characterized in that: The method is applicable when the image resolutions of the reference polarization image A and the polarization image B to be registered are the same.

3. The image registration method based on Sober edge detection according to claim 1, characterized in that: The feature extraction of the polarization image also includes preprocessing before feature extraction: Performing a color image-to-grayscale image conversion operation on the reference polarization image A and the polarization image B to be registered; For the reference image A, when it has a different image resolution from the polarization image B, it is necessary to perform a scaling operation on the image A, and by defining a scaling factor, the reference image A is scaled to have the same number of pixels and image size as the reference image B.

4. The image registration method based on Sober edge detection according to claim 1, characterized in that: The process of extracting features from the reference polarization image A and the polarization image B to be registered by the Sober edge detection algorithm is the same. The process of extracting features from the reference polarization image A by the Sober edge detection algorithm is as follows: The Sober edge detection algorithm is used to calculate the gradient information of the reference polarization image A pixel to perform edge detection operations and extract the feature information of the polarization image. The convolution formula of the Sober matrix is ​​as follows: Where: g = f*h represents the convolution operation on the image; h: represents the convolution kernel used in the Sober edge detection algorithm, which is used to calculate the gradient information of each pixel in the image. It extracts edge features by performing a convolution operation with the local area of ​​the image f; f: represents the original image, i.e., the reference polarization image A or the polarization image B to be registered; f(ik,jl): represents the result of multiplying the pixel value of the original image f at point f(ik,jl) by the weight value h(k,l) of the convolution kernel h at point (k,l). By multiplying each weight h(k,l) in the convolution kernel with the pixel value f(ik,jl) at the corresponding position in the image f. Among them, h(k,l) is the relative position coordinate in the convolution kernel, and (ik,jl) represents the absolute position coordinate in the image, which is determined by aligning the center of the convolution kernel with the pixel point in the image; ∑k,l: represents the sum of all k and l values ​​in the convolution kernel h. For each k and l, f(ik,jl) is the value of the pixel in image f aligned with the center of the convolution kernel, and h(k,l) is the weight of the convolution kernel at the position (k,l); i, j: the horizontal and vertical coordinates of the pixel point in the area with the maximum gradient value threshold in the polarization reference image A or the polarization image to be registered B, respectively; k, l: represent the horizontal and vertical coordinates of the position of the element in the convolution kernel h. During the convolution operation, k and l will traverse all elements of the convolution kernel and multiply each element of the convolution kernel with the pixel value of the corresponding position in the local area of ​​the image f, and finally sum them up to obtain the gradient information of the local area; The Sober convolution factor contains two sets of 3x3 matrices, one for horizontal and one for vertical. The two matrices are convolved with the polarization reference image A to obtain the approximate brightness difference in the horizontal and vertical directions. The polarization reference image A represents the original image, and Gx and Gy represent the gradient amplitudes detected by the horizontal and vertical edges, respectively. The formula is as follows: The reference polarization image A data is multiplied and added with the value of the operator at the corresponding position to obtain the horizontal gradient Gx and vertical gradient Gy of each pixel in the image. Then the gradient amplitude (G) and direction (θ) of each pixel are calculated. The specific formula is as follows: θ=ai.ctaii(GI@ / Gr)+(il@)rr Compare the calculated gradient amplitude (G) with the set threshold. If the gradient amplitude (G) is greater than the threshold, the point (f, h) is considered to be an edge point and displayed as a black point. Otherwise, it is displayed as a white point. The gradient amplitude is binarized to obtain the final edge image. When the amplitude G is less than or equal to the set threshold, it indicates that the edge strength of the pixel is weak.

5. The image registration method based on Sober edge detection according to claim 1, characterized in that: The process of performing maximum similarity point matching on the features extracted from the polarization reference image A and the features extracted from the polarization image to be registered B using the improved normalized cross-correlation matching method is as follows: The improved normalized cross-correlation matching method is used to find the pixel position with the maximum similarity. The calculation formula is as follows: The formula is used to measure the correlation or similarity between the polarization reference image and the polarization image to be registered. The numerator represents the sum of the products of the pixel values ​​of the polarization image and the polarization image to be registered, and the denominator represents the square root of the sum of the squares of the pixel values ​​of the polarization reference image and the polarization image to be registered. Where: value (R) represents the similarity between two pixel points, f(i,j) and g(i,j) represent the pixel values ​​of the area with the maximum gradient value threshold in the polarization reference image, where (i,j) is the two-dimensional coordinate of the pixel in the polarization image to be registered, i represents the row index, ranging from 1 to M; j represents the column index, ranging from 1 to N; M represents the height of the polarization reference image, that is, the number of rows of pixels in the polarization reference image; N represents the width of the polarization reference image, that is, the number of columns of pixels in the polarization reference image; H represents the height of the polarization image to be registered, that is, the number of rows of pixels in the polarization image to be registered; W represents the width of the polarization image to be registered, that is, the number of columns of pixels in the polarization image to be registered; The matching is completed according to the bidirectional matching strategy. The bidirectional matching strategy is that the two characteristic pixel points in the polarization reference image A and the polarization image to be registered B must satisfy the maximum correlation coefficient with each other to become a matching pair.

6. The image registration method based on Sober edge detection according to claim 1, characterized in that: The process of constructing the translation matrix through the correspondence relationship of the maximum similarity points and completing the construction of the translation parameter model M is as follows: 41): Assume that a certain pixel value similarity point set is C = {c1, c2, ... cs}, and randomly sample a pair of matching pairs S; 42) After removing a pair of randomly selected matching points from set C, calculate the root mean square error between the remaining feature matching point pairs (C') in set C after removing set S and set S; then repeat the above steps until the root mean square error between each matching point pair set S and the remaining feature matching point pairs (C') is obtained. If the root mean square error is less than the set threshold t, it is considered to be the optimal matching point pair for the pixel; 43) The steps for solving the root mean square error RMSE are as follows: assume that the maximum similarity value in the matching pair set S is the actual value, and the remaining feature matching point pairs (C') are the predicted values; then calculate the error value between the predicted value and the actual value, the sum of the squares of the error values, the mean square error (MSE) and the root mean square error (RMSE) in turn. If the root mean square error is lower than the threshold t, the matching point pair is considered to be the optimal matching point pair for the pixel point, which is S = {sm}, and the values ​​corresponding to the two pixel value similarity points in the matching point are f(i, j) and g(i, j); 44) Repeat steps 41)-43) until all the optimal matching point pairs are found. At this time, the matching pair set is S = {s1, s2, ... sm}, and it is used as the final translation parameter model M. Among them, the values ​​corresponding to the two pixel value similarity points in the i-th pair of matching points are f(i, j) and g(i, j).

7. The image registration method based on sober edge detection according to claim 1, characterized in that: The acquisition of the registration result specifically includes the following steps: 51) According to the position of the maximum pixel similarity point between the polarization reference image A and the polarization image to be registered B, the sub-pixel translation (tx i ,ty i ), and then find the translation amount between the polarization reference image A and the polarization image to be registered B; 52) constructing a translation matrix from the translation parameter model M; 53) The polarization reference image A and the polarization image to be registered B are translated and superimposed to complete the matching and obtain the matching result.

Images