Feature matching method for large-field-angle scene image

Abstract

The invention discloses a feature matching method for a large-field-angle scene image. The feature matching method mainly comprises the following steps of: (1) constructing an affine transformation space; projection of a camera visual angle according to a specified parameter is simulated, affine transformation is performed on a to-be-matched image, an affine space graph group is formed, an effective area of an affine image is determined, and a feature detection range is reduced. And (2) a feature detection stage. Inverse projection parameters are calculated through the projection parameters, feature detection of the affine image is achieved through AORB feature detection, feature points are projected to the original image through the inverse projection parameters, and anti-affine feature points are obtained. And (3) a description stage. On the basis of ORB description, comparison information of feature points and sampling points and gradient comparison information between sampling point pairs are added, a 3 * 256 binary descriptor is formed, and extraction and matching of the feature points of a large-view-angle scene are achieved.

Description

Technical Field

[0001] The invention relates to the field of photogrammetry, and in particular to an image feature matching method in a scene with a large field of view. Background Art

[0002] At present, the existing multi-scale feature detection methods are difficult to meet the requirements of feature detection and description in multi-affine scale space brought about by large viewing angle scenes, and face problems such as high computational complexity and failure due to strong parallax.

[0003] In 2011, Jean-Michel et al. published a paper titled "ASIFT: A new framework for fully affine invariant image comparison" in the journal SIAM journal on imaging sciences, proposing a feature extraction algorithm based on multi-scale affine transformation. In a sufficiently small local area, the algorithm uses affine transformation to eliminate the deformation caused by parallax, uses three parameters to describe the scale, longitude and latitude of the affine transformation, and constructs an affine transformation that simulates the perspective space. This algorithm solves the image deformation caused by perspective changes to a certain extent. The SIFT features used, combined with the three parameters of projection, can extract feature points with scale and affine invariance, which is suitable for image matching problems with large perspectives, strong rotations and affine transformations. The time complexity of feature extraction is greatly increased, which makes the algorithm relatively computationally expensive and not suitable for use in real-time applications. In addition, due to the harsh assumptions of the algorithm, it may not be applicable to some complex scenes.

[0004] In 2011, Alcantarilla et al. published a paper titled "Fast explicit diffusion for accelerated features in nonlinear scale spaces" in the journal IEEE Transactions on Pattern Analysis and Machine Intelligence, proposing a feature extraction algorithm based on nonlinear scale space, which uses a series of key technologies to improve scale invariance and robustness: ① Nonlinear scale space: The traditional scale space is based on the Gaussian pyramid, which uses linear scaling to simulate scale transformation. However, linear scale space is not robust enough for highly nonlinear image transformations (such as rotation and affine transformation). The KAZE algorithm uses nonlinear scale space to solve this problem, which uses nonlinear functions about scale to construct the scale space. This method can better adapt to nonlinear transformations and improve scale invariance. ② Fast feature point detection: The KAZE algorithm uses a fast feature point detection algorithm to extract feature points. ③ Feature descriptor: The KAZE algorithm uses a descriptor based on nonlinear scale space and second-order derivative operators, which can better handle rotation and affine transformations and improve the robustness of the descriptor. Compared with the SIFT algorithm, the feature descriptor of the KAZE algorithm has better performance and faster calculation speed.

[0005] In 2012, Alcantarilla et al. published a paper titled "KAZE features" at the Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, proposing the AKAZE (Accelerated-KAZE Features) algorithm. The AKAZE algorithm uses accelerated convolution technology to speed up calculations, thereby improving the algorithm's running speed. The AKAZE algorithm uses a nonlinear scalespace to improve the scale invariance of the algorithm. The AKAZE algorithm uses a new threshold strategy when detecting feature points, making it more robust to noise. The AKAZE algorithm uses a new descriptor algorithm to make it more accurate when matching feature points between different images.

[0006] In 2015, Yi et al. published a paper titled "Lift: Learned invariant feature transform" at the In: Computer Vision–ECCV 2016: 14th European Conference conference, proposing a feature point detector and descriptor LIFT. It is a method based on local binary patterns (LBP) that detects feature points by detecting the invariance of the LBP pattern and uses the information of the image gradient direction to describe the feature points. Due to the use of the gradient direction and the rotation invariance of the LBP pattern, LIFT has rotation and affine invariance. In addition, LIFT also has good performance and efficiency, and can be extended to feature point detection with larger scales and more directions.

[0007] However, Alcantarilla's KAZE and AKAZE algorithms and Yi's LIFT algorithm do not consider the perspective effect and affine deformation caused by viewing angle, which causes the algorithm to fail in a strong parallax environment. Summary of the invention

[0008] The present invention discloses a method for matching feature point images of scenes with a large field of view, which mainly includes: (1) constructing an affine transformation space stage. The camera viewing angle is simulated to be projected according to the specified parameters, and the image to be matched is affine transformed to form an affine space graph group, the effective area of ​​the affine image is determined, and the feature detection range is reduced. (2) Feature detection stage. The inverse projection parameters are calculated through the projection parameters, and the feature detection of the affine image is realized by using the AORB feature detection. The feature points are projected to the original image through the inverse projection parameters to obtain anti-affine feature points. (3) Description stage. On the basis of the ORB description, the comparison information between the feature points and the sampling points and the gradient comparison information between the sampling point pairs are added to form a 3*256 binary descriptor to realize the extraction and matching of feature points of scenes with a large viewing angle.

[0009] The present invention can be implemented by the following technical solutions:

[0010] S1: constructing the affine transformation space stage.

[0011] S1-1: Simulate the projection of the camera view according to the specified parameters.

[0012] (1) Building a camera imaging model

[0013] Perspective deformation caused by uneven surface of an object is not easy to handle. An object is regarded as composed of countless small planes with smooth enough surface, and its local perspective effect and nonlinear effect caused by projection transformation can be ignored. Approximately regard each small plane as a two-dimensional figure on a plane, and perform affine transformation on it, which is more convenient to handle and calculate. In the affine camera model, u represents the front view of the ideal smooth object plane, and l represents the camera plane.

[0014] The camera imaging model can be expressed by formula (1).

[0015] (1)

[0016] Among them, u is the target image, u0 is the front view, A is the affine transformation, G1 is the blur parameter, T and R are the translation and rotation parameters respectively, and S1 is the sampling value.

[0017] If any straight line in an image remains a straight line after being transformed into another image and can maintain a parallel relationship, then the transformation is an affine transformation.

[0018] (2)

[0019] Simplified formula (3)

[0020] (3)

[0021] Where OP is the output affine transformed image, IN is the input image to be transformed, A is the affine matrix, and B is the translation parameter.

[0022] Affine transformation has parallelism and straightness, which is specifically manifested in that the order of points on a straight line does not change after affine transformation, and the straight line remains a straight line. Affine transformation can be decomposed into rotation, translation, scaling, and inversion, expressed as:

[0023] (4)

[0024] Where ψ is the camera rotation angle; φ is the camera horizontal angle, λ is the distance parameter, and t is the eigenvalue of the affine matrix greater than 1. To describe the affine deformation of the vertical angle, a sampling parameter, namely the zenith angle, is added. The relationship between θ and t is shown in formula (5).

[0025] (5)

[0026] (2) Calculate affine transformation parameters

[0027] 1) Angle range

[0028] When simulating the change of viewing angle, it is easy to obtain from the upper limit of the horizontal rotation angle of the two-dimensional image that the upper limit of the horizontal angle is 180° and the upper limit of the altitude angle is 90°, that is, φ<180° and the altitude angle θ<90°.

[0029] 2) Parameter t

[0030] According to formula (5), the affine transformation is generally calculated using the parameter t in the affine matrix, and the upper limit of t is t max =4√2.

[0031] 3) t and φ sampling interval

[0032] According to formula (5), t is sampled using formula (6) and φ is sampled using formula (7).

[0033] (6)

[0034] Where = 5, n = √2

[0035] (7)

[0036] Where k is the number of sampling times and b is the sampling interval.

[0037] S1-2: Perform affine transformation on the image to be matched to form an affine space graph group.

[0038] After obtaining the affine transformation parameters, affine transformations with different parameters are performed on the images to be matched respectively, and an affine space graph group with matching images can be established.

[0039] S1-3: Determine the effective area of ​​the affine image and reduce the feature detection range.

[0040] The affine images of the affine space graph group all have large stretching and deformation. Therefore, the edge detection method of formula (8) is used to detect and determine the image range.

[0041] (8)

[0042] Where V img (x,y) is the binarization result of the pixel in row x and column y, and f(x,y) is the pixel value in row x and column y of the original affine image.

[0043] By traversing, detect the adjacent V img The number of rows and columns where (x,y) changes from 0 to 1 or from 1 to 0 is taken as the image range, denoted by E img .

[0044] S2: Feature detection stage.

[0045] When performing feature point detection, it is necessary to extract the feature points of the affine space graph group, and then perform corresponding inverse projection on the feature points of the affine image, project the feature points of the affine image into the original image, and obtain the feature points of the image to be matched. Among them, the feature point detection uses the AORB algorithm, which is easy to obtain a large number of corner points in a small range. It is necessary to use a non-maximum suppression algorithm to avoid the redundancy of feature points: for two adjacent corner points, calculate their respective score responses R, and retain the corner points with higher R values, where the score response R represents the sum of the absolute values ​​of the grayscale differences between the feature point and the 16 neighborhood sampling points.

[0046] S2-1: Use AORB feature detection to realize feature point detection of affine image.

[0047] (1) Extract a large number of feature points.

[0048] Select a point P from the image and determine whether it is a feature point: draw a circle with a radius of 3 pixels with P as the center. If the grayscale values ​​of n consecutive pixels on the circumference are greater or less than the grayscale value of point P, then P is determined to be a feature point. Generally, n is set to 12. In order to speed up the extraction of feature points and quickly exclude non-feature points, first check whether the grayscale values ​​at the 0°, 90°, 180° and 270° positions of the azimuth of point P are greater or less than the grayscale value of point P. If 3 or more pixel values ​​at the four positions are greater or less than the grayscale value of point P, then point P is determined to be a feature point. If not satisfied, the point is directly excluded.

[0049] (2) Use ID3 algorithm to extract the optimal feature points.

[0050] Use the ID3 algorithm to train a decision tree, and input the 16 pixels on the circumference of the feature point into the decision tree to screen out the optimal feature point.

[0051] (3) Non-maximum suppression removes local dense feature points.

[0052] Calculate the response size for each feature point. The calculation method is the sum of the absolute values ​​of the deviations between the feature point and the 16 feature points around it. Among the feature points that are close to each other, retain the feature points with larger values ​​and delete the rest.

[0053] (4) Scale invariance of feature points.

[0054] Construct a pyramid to achieve multi-scale invariance of feature points. That is, use the above steps (1)-(3), use the Fast algorithm to extract feature points on each layer of the pyramid, use the Harris corner point response function, sort by corner point response value, and select the first N feature points.

[0055] (5) Calculate the corner point orientation of the key points and describe the rotation invariance of the feature points.

[0056] The ORB algorithm proposes to use the grayscale centroid method to determine the direction of the FAST feature point. That is to say, the centroid of the feature point within the radius r is calculated by the moment, and the feature point coordinates to the centroid form a vector as the direction of the feature point.

[0057] First, define the moment of the pixel block in the neighborhood as m pq :

[0058] (9)

[0059] Where I represents the grayscale value at (x, y).

[0060] Then, the position C of the intensity center is calculated by formula (9), and the direction of the vector OC between it and the geometric center O is the main direction of the feature point.

[0061] (10)

[0062] (11)

[0063] S2-2: Calculate the inverse projection parameters through the projection parameters to obtain the anti-affine feature points.

[0064] According to the detector in the effective area of ​​the affine transformation space, the feature point information with scale σ, direction θ, and score response R is obtained, and the feature point information is substituted into formulas (1) to (5) to inversely calculate the inverse projection parameters. The feature points are projected to the original image through the inverse projection parameters to obtain anti-affine feature points.

[0065] S3: Description stage: Based on the description of the ORB algorithm, the comparison information between feature points and sampling points and the gradient comparison information between sampling point pairs are added to form a 3*256 binary descriptor to achieve the extraction and matching of feature points in large-viewing angle scenes.

[0066] S3-1: Characterization

[0067] The ORB algorithm increases the main direction of the feature points to achieve rotation invariance, and determines the selection of the optimal random sampling pairs in the neighborhood through statistical learning, thereby achieving the maximum discrimination of the descriptor.

[0068] (12)

[0069] In the optimal random sampling of 256 pairs of random sampling points P n (See Formula 12), calculate the grayscale contrast information between random sampling points, see Formula 13.

[0070] (13)

[0071] Among them, B n is the comparison result of the sampling point pair, and I is the grayscale value of any point. The results of all grayscale comparison information are aggregated to obtain a binary descriptor with a length of 256 as shown in formula (14).

[0072] (14)

[0073] Finally, a 256-dimensional binary descriptor in an affine space with rotation invariance is obtained, which realizes the description part of AORB.

[0074] S3-2: Characteristic value description

[0075] 1) Increase the comparison between feature points and sampling points

[0076] Add grayscale contrast information between feature points and sampling point pairs, see formulas (15) and (16).

[0077] (15)

[0078] (16)

[0079] By comparing the results between feature points and sampling point pairs, we take 1 if they are the same and 0 if they are different, and construct a 256-dimensional binary descriptor, thereby increasing the robustness and distinguishability of the feature points (see formula (17)).

[0080] (17)

[0081] 2) Add gradient contrast information

[0082] In order to better eliminate the deformation that cannot be eliminated by simulating the viewing angle, the multi-directional gradient of the sampling points is added on the basis of the above descriptors.

[0083] First, the gradient of the sampling point in each direction is calculated, and then the gradient of the sampling point in one direction is compared. If the gradient is the same, it is 1, and if it is different, it is 0, see formula (18).

[0084] (18)

[0085] Among them, g θ (x) indicates the sampling point x at The first-order gradient in the direction, see formula (19).

[0086] (19)

[0087] A 256-dimensional binary descriptor can be obtained in each direction, and finally different descriptor gradient descriptors are obtained according to the different number of gradient collections, see formula (20).

[0088] (20)

[0089] The ORB description part D1, the feature point and sampling point comparison part D2 and the multi-directional gradient part D3 are combined together to obtain the binary descriptor of the feature point, see formula (21).

[0090] (twenty one)

[0091] S3-3: Inverse affine transformation of descriptors

[0092] The descriptors obtained in the affine space are all for a certain affine image, and need to be combined with the corresponding inverse affine projection parameters of the affine image to perform inverse affine transformation and reflect it on the original image for subsequent feature matching. The feature points and feature descriptors after inverse affine transformation ultimately constitute the gradient-enhanced feature extraction and descriptors in the affine space.

[0093] S4: Image matching based on affine space

[0094] The FlannBasedMatcher image matching algorithm in opencv is used to match feature points, and the double minimum distance is used as the constraint condition to match the images to be matched.

[0095] Compared with the existing technology, the present invention has the following advantages: Alcantarilla's KAZE and AKAZE algorithms and Yi's LIFT algorithm do not consider the perspective effect and affine deformation caused by the viewing angle. The present invention simulates the viewpoint change of the target image to construct the affine transformation space of the image and obtains the affine invariance; the feature points are converted to the original image through the inverse affine transformation to obtain the feature points with anti-affine effect; the grayscale contrast information between the feature points and the sampling points and the gradient contrast information of the sampling points in multiple directions are added, a more accurate and fast binary descriptor is constructed, and a feature extraction algorithm based on the affine transformation space is proposed. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 It is a flow chart of the present invention

[0097] Figure 2 a (left picture) and b (right picture) of the image to be matched in the present invention

[0098] Figure 3 The camera imaging model of the present invention

[0099] Figure 4is the affine transformation space graph group of the image a to be matched in the present invention

[0100] Figure 5 is the affine transformation space graph group of the image b to be matched in the present invention

[0101] Figure 6 The image pair feature matching of the present invention is transformed into affine space graph group feature matching

[0102] Figure 7 The effective area determination of the affine image of the present invention is

[0103] Figure 8 The present invention reduces the detection pixel area

[0104] Fig. 9 is the simplified affine image group a of the present invention

[0105] Fig.10 is the feature extraction result of the first three layers of images of the affine image group a of the present invention

[0106] Fig.11 is the feature extraction result of the first three layers of images of the affine image group b of the present invention

[0107] Fig.12 The affine space image feature points of the present invention are inversely transformed to the original image

[0108] Fig.13 is the full affine feature point set of the image a to be matched in the present invention

[0109] Fig.14 The sampling window and its grayscale matrix of the present invention are

[0110] Fig.15 This is the ORB descriptor acquisition process of the present invention

[0111] Fig.16 This is the process of obtaining the comparative information between the feature points and the sampling points in the AORB descriptor of the present invention.

[0112] Fig.17 The multi-directional gradient contrast information acquisition process of the sampling points in the AGORB descriptor of the present invention is

[0113] Fig.18 The AGORB descriptor, AGORB1 descriptor, and AORB descriptor of the present invention are composed of

[0114] Fig.19 is the AGORB algorithm matching result of the present invention DETAILED DESCRIPTION

[0115] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, preferred embodiments are given below and the specific implementation of the present invention is further described in detail in conjunction with the accompanying drawings.

[0116] Example:

[0117] In a wide viewing angle scene, most images are taken at different viewing angles, and there is a certain parallax between different viewing angles. In order to make two images obtained at different viewing angles have more effective matching points, we can construct an affine space by constructing an affine space image group formed by a certain number of affine transformations of the two images, realize the parallax correction of the two images, and then obtain more feature points.

[0118] Combination Figure 1 The implementation process of this embodiment is described.

[0119] S1: constructing the affine transformation space stage.

[0120] S1-1: Simulate the projection of the camera view according to the specified parameters.

[0121] Combination Figure 2 The data used in this embodiment are images a (left image) and b (right image) to be matched, where a is the main view and b is the image rotated 25 degrees horizontally. The affine space construction process is as follows: ① Combine Figure 3 , build the camera imaging model, determine the affine space projection parameters t, Matrix; ② Taking the original image as an example, perform affine transformation with different parameters to obtain images with different perspectives.

[0122] Determine the affine space projection parameters t and , the obtained parameter list is saved in the following table, where when t is 1, 5 sets of projection parameters can be obtained, when t is 2, 5 sets of projection parameters can be obtained, and when t is 2 When t is 4, 8 sets of projection parameters can be obtained. , 15 sets of projection parameters can be obtained, as shown in Table 1 below.

[0123] Table 1 Affine space projection parameter list

[0124]

[0125] S1-2: Perform an affine transformation on the image to be matched to form an affine space image group. Perform an affine transformation on the original image pair using the parameters in Table 1 above to obtain an affine space image of the original image.

[0126] After the images a and b to be matched are transformed by the affine transformation of the parameters in Table 1 above, the affine space graph group shown in the figure below is formed, as shown in Figure 4 and Figure 5With different values ​​of parameter t, affine transformation images under different viewing angles are obtained, which are divided into 5 layers in total, namely 1, 4, 5, 8, 10, and 15 layers. It can be seen that when the number of layers increases, the deformation of the image becomes greater. When the deformation is greater, the affine image can cope with the image matching work with greater parallax changes in a large viewing angle.

[0127] By constructing the affine space graph group for the matching objects a and b with the parameters in the parameter list in Table 1, the radiographic images of the images at various viewing angles can be restored, so that the deformation originally caused by parallax during feature extraction and matching can be weakened to a certain extent, and more feature point pairs can be extracted, that is, the feature extraction and matching process is introduced from the image pair to be matched into the affine space image pair, such as Figure 6 As shown, the images a and b to be matched and their affine space graph group are shown.

[0128] S1-3: Determine the effective area of ​​the affine image and reduce the feature detection range.

[0129] In the process of feature extraction, every pixel in the image will participate in the detection and extraction of feature points. The effective area of ​​the affine space image is determined, which is conducive to reducing the total pixels of feature detection, thereby achieving a certain improvement in detection efficiency.

[0130] 1) The affine transformed image obtained by affine transformation with different projection transformation parameters has different pixel effective areas, such as Figure 6 , and the specific redundant size is related to the projection parameters. Taking the image a to be matched as an example, the simplified effective area can be obtained by calculation, see Figure 7 .

[0131] Figure 7 The lower middle part is the affine space graph group of the image a to be matched, and the upper left part is a detailed picture of the second affine image of the second layer of the affine space graph group. It can be seen that after the affine transformation, the image has undergone certain rotation and scale changes compared to the original image. At the same time, it can be seen that there are certain image stretching areas outside the content of the original image, such as the upper left corner, upper right corner, lower left corner, and lower right corner.

[0132] 2) Determine the effective area and remove redundant pixels of the affine image group. By determining the effective area and removing redundant pixels of each radiological image, a simplified affine image can be obtained. Figure 8 Take the image a to be matched as an example, its affine image group is shown in Fig. 9 .

[0133] S2: Feature detection stage.

[0134] S2-1: Use AORB feature detection to realize feature point detection of affine image.

[0135] Based on the affine space graph group of the images a and b to be matched, ORB features are extracted for each affine image in the graph group, and the inverse affine transformation parameters are calculated according to the affine transformation parameters of the image. After the feature points are obtained, the position and main direction of the feature points are inversely transformed using the inverse reflection transformation parameters to return them to the original image, and finally the feature point set of the original image can be obtained.

[0136] Based on the simplified affine image group a, ORB feature point detection is used to extract feature points of the affine transformed images under different projections at one time, and the following can be obtained: Fig.10 and Fig.11 The feature points shown in Fig. 1 are the affine space graphs of the image a to be matched (see Fig.10 ) are marked with yellow lines, and the affine space graph group of the image b to be matched (see Fig.11 ) are marked with green lines. Since there are many projection methods for affine graph groups, only the first layer of the affine space ( Fig.10 (a) and Fig.11 (a)), the second layer ( Fig.10 (b) to (e) and Fig.11 (b) to (e)), the third layer ( Fig.10 (f) and Fig.11 (f) Feature points obtained from the image.

[0137] S2-2: Calculate the inverse projection parameters through the projection parameters to obtain the anti-affine feature points.

[0138] The projection parameters corresponding to each group of images in the affine space can affine transform the original image into an affine image under a new perspective. At the same time, the feature points extracted from the affine image under the new perspective can also be quasi-affine transformed through the inverse affine transformation parameters, thereby realizing feature point extraction in the affine space and realizing the anti-affine capability of feature point extraction.

[0139] The inverse affine transformation parameters correspond to the affine transformation parameters one by one. According to the different projection parameters t, they are divided into five layers. Each layer is based on The inverse affine transformation parameters are composed of a 2×3 matrix, with a total of 6 parameters, as shown in Table 4.2, where t=4, 4 Only some parameters are displayed.

[0140] Table 2 List of inverse affine space projection parameters

[0141]

[0142] Combination Fig.12, which illustrates the process of inverse affine transformation of feature points in affine space image to the original image. On the left is the image a to be matched and its total feature point set. The six arcs on the right represent the six different layers of the affine space image group, from the first layer to the sixth layer from left to right. The images on the arcs are the affine images contained in the layer and the ORB feature points of the image. In the middle is an affine image of the second layer, and the inverse affine transformation parameters corresponding to its affine transformation parameters are in the middle. The feature points in the 43 affine images in the 6 layers corresponding to each image to be matched will all be on the image to be matched after the inverse affine transformation, and finally constitute the feature points of the image A to be matched, as shown in Fig.13 .

[0143] S3: Description stage.

[0144] S3-1: ORB feature description.

[0145] Taking the image to be matched as an example, a window with a neighborhood size of 31×31 is intercepted in the image to be matched, and the following can be obtained: Fig.14 Grayscale window (left) and part of the grayscale matrix (right).

[0146] ORB uses the improved BRIEF descriptor to describe features. This method can be used to obtain the ORB descriptor of a feature point, which can be described as a 32-dimensional vector, where the vector size is 0 to 255, and each value corresponds to an 8-bit binary vector, which together constitute the descriptor of a feature point. Taking the descriptor of a feature point on an affine layer image of the image to be matched a as an example, through Fig.15 The process can obtain the ORB descriptor in Table 3.

[0147] Table 3 ORB description subtype and content

[0148]

[0149] S3-2: Characteristic value description

[0150] In order to cope with the image deformation caused by affine transformation, multi-directional gradient contrast information is used to enhance the anti-affine ability of feature points. The calculation process of the descriptor includes two processes. One is to solve the contrast information between feature points and sampling points; the other is to solve the multi-directional gradient contrast information of sampling points.

[0151] ① When the position of the feature point p is determined, the neighborhood interval is determined, and the sampling point pair is determined, the comparison information between the feature point and the sampling point can be solved to obtain the comparison part of the feature point and the sampling point of the AGORB descriptor. Fig.16 .

[0152] Fig.16Taking the random sampling point pair p1, p2 as an example, first calculate the gray value comparison information of the feature point p and the random sampling point pair p1, p2. When the gray value comparison results of the feature point p and the feature points p1, p2 are the same, take 1, otherwise take 0. This part can obtain a descriptor with a length of 256, which is used as the second part of the AGORB descriptor.

[0153] ② Sampling points for multi-directional gradient contrast information

[0154] In order to cope with the influence of affine deformation, after the above 256 pairs of random sampling points are determined, the AGORB descriptor also needs to add the multi-directional gradient contrast information of the sampling points on the basis of the above descriptors. For details, see the process. Fig.17 .

[0155] Fig.17 Take the random sampling point pair p1, p2 as an example. When , calculate the horizontal gradient of the sampling point to p1 and p2, and compare the results. If they are the same, take 1, otherwise take 0. This part can obtain a descriptor with a length of 256; repeat , the descriptors of each direction can be obtained respectively.

[0156] when When , the ORB description part, the feature value and sampling point grayscale comparison part and the four-directional gradient comparison information part are combined to form the AGORB descriptor of this paper; when When , the ORB description part, the feature value and the sampling point grayscale contrast part and the two-directional gradient contrast information part are combined to form the AGORB1 descriptor of this paper; without considering the gradient contrast information, only the ORB description part, the feature value and the sampling point grayscale contrast part are combined to form the AORB descriptor. Fig.18 .

[0157] S4: Image matching based on affine space

[0158] Taking the images a and b to be matched as examples, the feature point set of the images to be matched taking into account the affine transformation is obtained by constructing the affine space graph group of the images to be matched, extracting the feature points in the affine space, and performing the inverse affine transformation of the feature points. By combining ORB feature description and eigenvalue description, a binary descriptor is constructed. The FlannBasedMatcher image matching algorithm in opencv is used to match the feature points, and the two times minimum distance is used as the constraint condition to match the images a and b to be matched. The results are shown below. Fig.19 shown.

[0159] The above embodiments are only used to illustrate the present invention, but not to limit the present invention. A person skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the present invention, and the patent protection scope of the present invention shall be defined by the claims.

[0160] The technical contents not described in detail in the present invention are all known technologies.

Claims

1. Feature point image matching method for scenes with large field of view, characterized in that It mainly includes: (1) constructing the affine transformation space stage; simulating the projection of the camera view according to the specified parameters, performing affine transformation on the image to be matched, forming an affine space graph group, determining the effective area of ​​the affine image, and reducing the feature detection range; (2) Feature detection stage: The inverse projection parameters are calculated through the projection parameters, and the feature detection of the affine image is realized by using the AORB feature detection. The feature points are projected onto the original image through the inverse projection parameters to obtain the anti-affine feature points. (3) Description stage: Based on the ORB description, the contrast information between the feature points and the sampling points and the gradient contrast information between the sampling point pairs are added to form a 3*256 binary descriptor to realize the extraction and matching of feature points of the large viewing angle scene. The invention is also characterized in that the following technical solutions are adopted: S1: Constructing the affine transformation space stage S1-1: Simulate the projection of the camera view according to the specified parameters (1) Building a camera imaging model Perspective deformation caused by uneven surface of an object is not easy to handle. An object is regarded as composed of countless small planes with smooth enough surface, and its local perspective effect and nonlinear effect caused by projection transformation can be ignored; each small plane is approximately regarded as a two-dimensional figure on a plane, and affine transformation is performed on it, which is more convenient to handle and calculate; in the affine camera model, u represents the front view of the ideal smooth object plane, and l represents the camera plane; The camera imaging model can be expressed by formula (1); （1）； Among them, u is the target image, u0 is the front view, A is the affine transformation, G1 is the blur parameter, T and R are the translation and rotation parameters respectively, and S1 is the sampling value; If any straight line in an image remains a straight line in another image after being transformed and can maintain a parallel relationship, then the transformation is an affine transformation. （2）； Simplified formula (3) （3）； Where OP is the output affine transformed image, IN is the input image to be transformed, A is the affine matrix, and B is the translation parameter; Affine transformation has parallelism and straightness, which is specifically manifested in that the order of points on a straight line does not change after affine transformation, and the straight line remains a straight line; affine transformation can be decomposed into rotation, translation, scaling, and inversion, expressed as: （4）； Among them, ψ is the camera rotation angle; φ is the camera horizontal angle, λ is the distance parameter, and t is the eigenvalue of the affine matrix greater than 1; in order to describe the affine deformation of the vertical angle, a sampling parameter, namely the zenith angle, is added; the relationship between θ and t is shown in formula (5); （5）； (2) Calculate affine transformation parameters 1) Angle range When simulating the change of viewing angle, it is easy to obtain the upper limit of the horizontal angle of 180° and the upper limit of the altitude angle of 90° from the upper limit of the horizontal rotation angle of the two-dimensional image, that is, φ<180° and the altitude angle θ<90°; 2) Parameter t According to formula (5), the affine transformation is generally calculated using the parameter t in the affine matrix, and the upper limit of t is t max =4√2; 3) t and φ sampling interval According to formula (5), t is sampled using formula (6) and φ is sampled using formula (7); （6）； Where = 5, n = √2 （7）； Where k is the number of sampling times and b is the sampling interval; S1-2: Perform affine transformation on the image to be matched to form an affine space graph group After obtaining the affine transformation parameters, affine transformations with different parameters are performed on the images to be matched, and an affine space graph group with matching images can be established; S1-3: Determine the effective area of ​​the affine image and reduce the feature detection range The affine images of the affine space graph group all have large stretching and deformation, so the edge detection method of formula (8) is used to detect and determine the image range; （8）； Where V img (x, y) is the binarization result of the pixel in row x and column y, and f(x, y) is the pixel value in row x and column y of the original affine image; By traversing, detect the adjacent V img The number of rows and columns where (x,y) changes from 0 to 1 or from 1 to 0 is taken as the image range, denoted by E img ; S2: Feature detection stage When performing feature point detection, it is necessary to extract the feature points of the affine space graph group, and then perform corresponding inverse projection on the feature points of the affine image, project the feature points of the affine image into the original image, and obtain the feature points of the image to be matched; Among them, the feature point detection adopts the AORB algorithm, which is easy to obtain a large number of corner points in a small range, and it is necessary to use the non-maximum suppression algorithm to avoid the redundancy of feature points: for two adjacent corner points, calculate their respective score responses R, and retain the corner points with higher R values, where the score response R represents the sum of the absolute values ​​of the grayscale differences between the feature point and the 16 neighborhood sampling points; S2-1: Feature point detection of affine images using AORB feature detection (1) Extracting a large number of feature points Select a point P from the image and determine whether it is a feature point: draw a circle with a radius of 3 pixels with P as the center; if there are n consecutive pixel points on the circumference with grayscale values ​​greater or less than the grayscale value of point P, then P is determined to be a feature point; n is generally set to 12; to speed up the extraction of feature points and quickly exclude non-feature points, first detect whether the grayscale values ​​at the azimuth positions of 0°, 90°, 180° and 270° of point P are greater or less than the grayscale value of point P. If 3 or more pixel values ​​at the four positions are greater or less than the grayscale value of point P, then point P is determined to be a feature point; if not satisfied, then the point is directly excluded; (2) Extract the optimal feature points using the ID3 algorithm Use the ID3 algorithm to train a decision tree, and input the 16 pixels on the circumference of the feature point into the decision tree to screen out the optimal feature point; (3) Non-maximum suppression to remove local dense feature points Calculate the response size for each feature point; the calculation method is the sum of the absolute values ​​of the deviations between the feature point and the 16 feature points around it; among the feature points that are relatively close, retain the feature points with larger values ​​and delete the rest; (4) Scale invariance of feature points Construct a pyramid to achieve multi-scale invariance of feature points. That is, use the above steps (1)-(3) to extract feature points on each layer of the pyramid using the Fast algorithm, use the Harris corner point response function, sort by corner point response value, and select the first N feature points; (5) Calculate the corner point direction of the key point and describe the rotation invariance of the feature point The ORB algorithm proposes to use the grayscale centroid method to determine the direction of the FAST feature point; that is, the centroid of the feature point within the radius r is calculated by the moment, and the feature point coordinates to the centroid form a vector as the direction of the feature point; First, define the moment of the pixel block in the neighborhood as m pq : （9）； Where I represents the gray value at (x, y); Then, the position C of the intensity center is calculated by formula (9), and the direction of the vector OC between it and the geometric center O is the main direction of the feature point; （10）； （11）； S2-2: Calculate the inverse projection parameters through the projection parameters to obtain the anti-affine feature points According to the detector in the effective area of ​​the affine transformation space, feature point information with scale σ, direction θ, and score response R is obtained, and the feature point information is substituted into formulas (1) to (5) to inversely calculate the inverse projection parameters; the feature points are projected to the original image through the inverse projection parameters to obtain anti-affine feature points; S3: Description stage On the basis of the description of the ORB algorithm, the comparison information between feature points and sampling points and the gradient comparison information between sampling point pairs are added to form a 3*256 binary descriptor to achieve the extraction and matching of feature points in large viewing angle scenes. S3-1: Characterization The ORB algorithm increases the main direction of the feature points to achieve rotation invariance, and determines the selection of the optimal random sampling pair in the neighborhood through statistical learning, achieving the maximum discrimination of the descriptor; （12）； In the optimal random sampling of 256 pairs of random sampling points P n (See formula 12), calculate the grayscale contrast information between random sampling points, see formula (13); （13）； Among them, B n is the comparison result of the sampling point pair, I is the gray value of any point; and the results of all gray comparison information are collected to obtain a binary descriptor with a length of 256 as shown in formula (14); （14）； Finally, a 256-dimensional binary descriptor in an affine space with rotation invariance is obtained, which realizes the description part of AORB. S3-2: Characteristic value description 1) Increase the comparison between feature points and sampling points Add grayscale contrast information between feature points and sampling point pairs, see formulas (15) and (16); （15）； （16）； By comparing the comparison results between the feature points and the sampling point pairs, the same ones are taken as 1, and the different ones are taken as 0, and a 256-dimensional binary descriptor is constructed, thereby increasing the robustness and distinguishability of the feature points (see formula (17)); （17）； 2) Add gradient contrast information In order to better eliminate the deformation that cannot be eliminated by simulating the view angle, multi-directional gradients of sampling points are added on the basis of the above descriptors; First, the gradients of the sampling points in each direction are calculated, and then the gradients of the sampling points in one direction are compared; if they are the same, they are taken as 1, and if they are different, they are taken as 0, see formula (18); （18）； Among them, g θ (x) indicates the sampling point x at The first-order gradient in the direction, see formula (19); （19）； A 256-dimensional binary descriptor can be obtained in each direction, and finally different descriptor gradient descriptors are obtained according to the different number of gradient collections, as shown in formula (20); （20）； The ORB description part D1, the feature point and sampling point comparison part D2 and the multi-directional gradient part D3 are combined together to obtain the binary descriptor of the feature point, see formula (21); （21）； S3-3: Inverse affine transformation of descriptors The descriptors obtained in the affine space are all for a certain affine image, and need to be combined with the corresponding inverse affine projection parameters of the affine image to perform inverse affine transformation and reflect it on the original image for subsequent feature matching work; the feature points and feature descriptors after inverse affine transformation ultimately constitute the gradient-enhanced feature extraction and descriptors in the affine space; S4: Image matching based on affine space The FlannBasedMatcher image matching algorithm in opencv is used to match feature points, and the double minimum distance is used as the constraint condition to match the images to be matched.

Images