A method and device for feature extraction and matching of satellite video images

Abstract

The present invention discloses a method and device for feature extraction and matching of satellite video images, which can solve the problem of real-time or near-real-time application of satellite video. The satellite video image to be matched is determined, corner points are extracted from the satellite video image to be matched; the corner points are filtered; based on the filtered corner points, the satellite video image to be matched is characterized, and a feature vector representing the satellite video image to be matched is obtained; and the feature vectors of the satellite video image to be matched are matched.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and device for feature extraction and matching of satellite video images. Background Art

[0002] Feature extraction and matching is one of the important prerequisites for the application of Computer Vision (CV). A feature is an attribute that can distinguish a target from other targets. Common features include point features, line features, surface features, and even volume features. Mathematically, matching can be understood as a problem of measuring similarity between high-dimensional vectors through a distance function. The choice of distance metric function is one of the key factors affecting matching efficiency and accuracy. For different descriptors, there are often different distance metric functions that can be used. In the field of remote sensing, matching pairs of points with the same name can be used for applications such as band registration of images, stabilization of satellite videos, and pose estimation between satellite video frames.

[0003] Prior art 1 (see Lowe, David G. "Object recognition from local scale-invariant features". Proceedings of the International Conference on Computer Vision. 2. pp. 1150–1157.) provides a method for feature extraction and matching, which mainly includes four parts: extreme value detection in the image scale space, precise positioning of feature corners, feature direction determination, and feature point description. Feature point detection is achieved based on the Gaussian difference pyramid, and feature point description is achieved using the grayscale direction histogram, and a high-dimensional floating-point descriptor is used for matching. The application of SIFT features in inter-frame feature extraction and matching of satellite videos is mainly due to efficiency issues. Since it is a floating-point descriptor, Euclidean distance is generally used as the metric function, and KD-Tree tree construction is used to accelerate matching. However, the above acceleration methods cannot meet the scenarios with large amounts of satellite video data and real-time or near real-time requirements.

[0004] Prior art 2 (see Rublee, Ethan; Rabaud, Vincent; Konolige, Kurt; Bradski, Gary. "ORB: an efficient alternative to SIFT or SURF" (PDF). IEEE International Conference on Computer Vision (ICCV).) provides a method for fast feature extraction and matching. By adopting Oriented-FAST and Steer-BRIEF, only the grayscale centroid of the window and logical judgment need to be calculated. The algorithm is simple and basically meets the real-time requirements. However, this will also lead to a certain degree of accuracy loss. Since the FAST corner point detection operator has a strong edge response, it is easy to find many pseudo feature points on the edge. To address this problem, ORB calculates the Harris score of each candidate corner point and sorts them, and finally selects the first n corner points as feature points through a threshold. However, the quality of the feature points extracted in this way is greatly affected by the threshold. If the threshold is set too low, some strongly responsive edge "pseudo corner points" will be considered as corner points, resulting in a "clustering" phenomenon when extracting a large number of ORB feature points, that is, the feature points are unevenly distributed. Clustered corner points will have a great impact on subsequent matching and pose estimation. Although a large number of feature points are extracted, the number of feature point pairs that are finally matched is small, and due to the uneven distribution of feature points, the overall changes cannot be reflected.

[0005] Considering the above factors, the present invention first uses the Gaussian-Laplacian operator to detect and extract the corner points of the satellite video image, and then obtains the corner points to be selected. The feature point screening method is used to filter non-corner points such as low-contrast corner points and pseudo-edge response points to achieve accurate detection of feature points. After obtaining the corner points, the corner points are described using a multidimensional binary descriptor to obtain a feature description vector of the corner points. Finally, the feature vector is similarly measured using a metric function and a matching algorithm to obtain matching pairs of points with the same name. This method has the characteristics of small computational complexity, high feature point extraction accuracy, and wide applicability. It is suitable for applications such as inter-frame stabilization, inter-frame pose estimation, and inter-frame target tracking of satellite videos. The solution of the present invention can quickly extract and match features for large amounts of satellite video data, and is suitable for application requirements and scenarios for rapid feature extraction and matching between satellite video frames. Summary of the invention

[0006] In view of this, the present invention provides a method and device for feature extraction and matching of satellite video images, which can solve the real-time or near real-time application problem of satellite video, provide starting data and results for further real-time or near real-time application of satellite video, and has the characteristics of wide application fields.

[0007] In order to solve the above technical problems, the present invention is implemented as follows.

[0008] A method for feature extraction and matching of satellite video images, comprising:

[0009] Step S1: determining a satellite video image to be matched, and extracting corner points from the satellite video image to be matched;

[0010] Step S2: determining the difference between the gray value of each corner point and the gray value of the extreme point corresponding to the corner point, comparing the difference with a preset threshold, and performing a first filtering on the corner points; for each corner point after the first filtering, determining the stability of the corner point; and performing a second filtering on the corner points based on the stability;

[0011] Step S3: Based on the corner points after secondary filtering, characterize the satellite video image to be matched, and obtain a feature vector characterizing the satellite video image to be matched;

[0012] Step S4: Matching the feature vectors of the satellite video images to be matched.

[0013] Preferably, the step S1 specifically includes:

[0014] Step S101: Divide the satellite video image to be matched into a plurality of image blocks I ij , extract several feature points from each image block, image block I ij Represents the image block in row i and column j;

[0015] Step S102: constructing a Gaussian-Laplacian operator, and using the Gaussian-Laplacian operator to extract a predetermined number of corner points from each image block, wherein:

[0016] The Gaussian-Laplacian operator is

[0017]

[0018] Among them, LoG (Laplacian of Gaussian) is the Gaussian-Laplacian operator, which describes the two-dimensional isotropic measure of the second-order spatial derivative of the image, ΔG σ (x, y) is the second-order derivative of the Gaussian convolution function at this point, x is the image pixel column coordinate, y is the image pixel row coordinate, G σ (x, y) is the Gaussian convolution function, σ is the standard deviation of the Gaussian filter function;

[0019]

[0020] Where k is 1.6, G(x, y, kσ) and G(x, y, σ) are Gaussian filtering results with parameters σ and kσ respectively;

[0021] Use the Gaussian-Laplacian operator to extract a predetermined number of corner points from each image block that has been segmented, calculate and record the image block I ij Each (x, y) that maximizes Formula 4 is:

[0022]

[0023] Among them, f(x,y) is the gray value of point (x,y); the obtained (x,y) constitutes the corner points initially extracted from the satellite video image.

[0024] Preferably, the step S2: filtering the corner points comprises:

[0025] Step S201: determine and filter low-contrast corner points, that is, remove candidate points that are sensitive to noise:

[0026] For each candidate corner point, let the candidate corner point be p0(x,y), and the corresponding extreme point The offset is defined as Δp(Δx, Δy), and the absolute value of the difference between the extreme point and the corner point to be selected is calculated. Take the gray value threshold Th=0.03, if Diff≤Th, then keep the candidate corner point; otherwise, consider that the candidate corner point is unstable and filter the candidate corner point; The extreme point The gray value of , D(p0) is the gray value of the corner point p0 to be selected;

[0027] Step S202: Determine and delete unstable edge response points from the filtered candidate corner points, that is, perform the following operations on each of the filtered candidate corner points:

[0028] For the filtered candidate corner point p′(x,y), construct the Hessian matrix H of the filtered candidate corner point

[0029]

[0030] Among them, D xx , D xy , D yy is the difference result of the filtered candidate contact point p′(x,y) field position;

[0031] Let α be the maximum eigenvalue of the H matrix, β be the minimum eigenvalue, and α and β can represent the gradients in the x and y directions respectively, then:

[0032] Tr(H)=D xx +D yy =α+β

[0033] Det(H)=D xx Dyy -(D xy ) 2 =αβ

[0034] Set the threshold r,

[0035]

[0036] If the candidate corner point p′(x, y) meets the judgment condition, the candidate corner point is an unstable edge response point and is deleted; otherwise, the candidate corner point is retained.

[0037] Preferably, the step S3 comprises:

[0038] Filter to get the corner point set Point = {p1,…,p i ,…p num}, where 1≤i≤num, num is the number of corner points in the filtered corner point set;

[0039] For the corner point p i , select the corner point p i The surrounding 128 pairs of pixels inum1 ,Q inum1},S inum1 , Q inum1 is the corner point p i The surrounding pixels, and S inum1 , Q inum1 The distribution law conforms to the Gaussian distribution; each pixel pair uses a binary descriptor R inum1 To characterize,

[0040] Among them, 1≤num1≤128, then the corner point p i It can be represented by a 128-dimensional feature vector composed of 0 and 1; at this time, the feature vector of the satellite video image to be matched is obtained, which is composed of the corner point p i It is composed of the size relationship between the surrounding 128 groups of pixel pairs.

[0041] Preferably, step S4: matching the feature vectors of the satellite video images to be matched, wherein:

[0042] The satellite video images to be matched are all represented by binary vectors of equal length. The binary vectors are used as binary strings to treat the satellite video images I a ,I b The binary strings corresponding to the feature points in the Calculate Hamming distances between each pair, that is

[0043]

[0044] Among them, s represents the sth value of the feature vector, dist h Represents the Hamming distance, dist s Calculated as follows:

[0045]

[0046] Among them, a s and b s I a and I b The value of the feature vector corresponding to the position of the feature point in .

[0047] When the Hamming distance dist h When it is greater than 0.8 times the vector length, it is considered that the satellite video image to be matched is I a ,I b This set of feature points in matches; otherwise, they do not match.

[0048] The present invention provides a device for extracting and matching features of satellite video images, the device comprising:

[0049] Corner point acquisition module: configured to determine the satellite video image to be matched, and extract corner points from the satellite video image to be matched;

[0050] Corner point filtering module: configured to determine the difference between the gray value of each corner point and the gray value of the extreme point corresponding to the corner point, compare the difference with a preset threshold, and perform a first filtering on the corner points; for each corner point after the first filtering, determine the stability of the corner point; and perform a second filtering on the corner points based on the stability;

[0051] A feature vector acquisition module: configured to characterize the satellite video image to be matched based on the corner points after secondary filtering, and obtain a feature vector characterizing the satellite video image to be matched;

[0052] Matching module: configured to match feature vectors of satellite video images to be matched.

[0053] Preferably, the corner point acquisition module includes:

[0054] Image segmentation submodule: configured to divide the satellite video image to be matched into a plurality of image blocks I ij , extract several feature points from each image block, image block I ij Represents the image block in row i and column j;

[0055] Corner point acquisition submodule: configured to construct a Gaussian-Laplacian operator, and use the Gaussian-Laplacian operator to extract a predetermined number of corner points from each image block, wherein:

[0056] The Gaussian-Laplacian operator is

[0057]

[0058] Among them, LoG (Laplacian of Gaussian) is the Gaussian-Laplacian operator, which describes the two-dimensional isotropic measure of the second-order spatial derivative of the image, ΔG σ (x, y) is the second-order derivative of the Gaussian convolution function at this point, x is the image pixel column coordinate, y is the image pixel row coordinate, G σ (x, y) is the Gaussian convolution function, σ is the standard deviation of the Gaussian filter operator;

[0059]

[0060] Where k is 1.6, G(x, y, kσ) and G(x, y, σ) are Gaussian filtering results with parameters σ and kσ respectively;

[0061] Use the Gaussian-Laplacian operator to extract a predetermined number of corner points from each image block that has been segmented, calculate and record the image block I ij Each (x, y) that maximizes Formula 4 is:

[0062]

[0063] Among them, f(x,y) is the gray value of point (x,y); the obtained (x,y) constitutes the corner points initially extracted from the satellite video image.

[0064] Preferably, the corner point filtering module includes:

[0065] The first filtering submodule is configured to determine and filter low-contrast corner points, that is, to remove candidate points that are sensitive to noise:

[0066] For each candidate corner point, let the candidate corner point be p0(x,y), and the corresponding extreme point The offset is defined as Δp(Δx, Δy), and the absolute value of the difference between the extreme point and the corner point to be selected is calculated. Take the gray value threshold Th=0.03, if Diff≤Th, then keep the candidate corner point; otherwise, consider that the candidate corner point is unstable and filter the candidate corner point; The extreme point The gray value of , D(p0) is the gray value of the corner point p0 to be selected;

[0067] The second filtering submodule is configured to determine and delete unstable edge response points from the filtered candidate corner points, that is, for each of the filtered candidate corner points, perform the following operations:

[0068] For the filtered candidate corner point p′(x,y), construct the Hessian matrix H of the filtered candidate corner point

[0069]

[0070] Among them, D xx , D xy , D yy is the difference result of the filtered candidate contact point p′(x,y) field position;

[0071] Let α be the maximum eigenvalue of the H matrix, β be the minimum eigenvalue, and α and β can represent the gradients in the x and y directions respectively, then:

[0072] Tr(H)=D xx +D yy =α+β

[0073] Det(H)=D xx D yy -(D xy ) 2 =αβ

[0074] Set the threshold r,

[0075]

[0076] If the candidate corner point p′(x, y) meets the judgment condition, the candidate corner point is an unstable edge response point and is deleted; otherwise, the candidate corner point is retained.

[0077] Preferably, the feature vector acquisition module includes:

[0078] Corner point set acquisition submodule: configured to filter and obtain the corner point set Point = {p1, ..., p i ,…p num}, where 1≤i≤num, num is the number of corner points in the filtered corner point set;

[0079] Feature acquisition submodule: configured for corner point p i , select the corner point p i The surrounding 128 pairs of pixels inum1 ,Q inum1},S inum1 , Q inum1 is the corner point p i The surrounding pixels, and S inum1 , Q inum1 The distribution law conforms to the Gaussian distribution; each pixel pair uses a binary descriptor R inum1 To characterize,

[0080] Among them, 1≤num1≤128, then the corner point p i It can be represented by a 128-dimensional feature vector composed of 0 and 1; at this time, the feature vector of the satellite video image to be matched is obtained, which is composed of the corner point p i It is composed of the size relationship between the surrounding 128 groups of pixel pairs.

[0081] Preferably, the matching module, wherein:

[0082] The satellite video images to be matched are all represented by binary vectors of equal length. The binary vectors are used as binary strings to treat the satellite video images I a ,I b The binary strings corresponding to the feature points in the Calculate Hamming distances between each pair, that is

[0083]

[0084] Among them, s represents the sth value of the feature vector, dist h Represents the Hamming distance, dist s Calculated as follows:

[0085]

[0086] Among them, a s and b s I a and I b The value of the feature vector corresponding to the position of the feature point in .

[0087] When the Hamming distance dist h When it is greater than 0.8 times the vector length, it is considered that the satellite video image to be matched is I a ,I b This set of feature points in matches; otherwise, they do not match.

[0088] Beneficial effects:

[0089] (1) The present invention discloses a fast feature extraction and matching method suitable for satellite videos, which adopts a corner point detection algorithm based on pixel grayscale gradient and a more stringent corner point screening algorithm. Compared with ORB features, the extracted feature points are more accurate and reliable.

[0090] (2) The present invention discloses a fast feature extraction and matching method suitable for satellite videos, which uses a more efficient binary feature descriptor. Compared with the floating-point descriptor of SIFT features, its description, storage, and matching efficiency are higher and less time-consuming.

[0091] (3) The present invention discloses a fast feature extraction and matching method suitable for satellite videos, which is suitable for scenes such as inter-frame image stabilization and inter-frame pose estimation of satellite videos and has strong application capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 This is a flowchart of feature extraction and matching of satellite video images;

[0093] Figure 2 This is a schematic diagram of feature extraction and matching of satellite video images;

[0094] Figure 3 Schematic diagram of the extracted corner points to be selected;

[0095] Figure 4 Schematic diagram of corner points after screening;

[0096] Figure 5 Schematic diagram of matching feature point pairs obtained by matching;

[0097] Figure 6 Schematic diagram of the device structure for feature extraction and matching of satellite video images. DETAILED DESCRIPTION

[0098] The present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0099] The present invention first detects and extracts corner points of satellite video images by using Gaussian-Laplacian operator, and then obtains the corner points to be selected. The feature point screening method is used to filter non-corner points such as low-contrast corner points and pseudo-edge response points to achieve accurate detection of feature points. After obtaining the corner points, the corner points are described by multidimensional binary descriptors to obtain feature description vectors of the corner points. Finally, the feature vectors are similarity measured by using metric functions and matching algorithms to obtain matching pairs of points with the same name.

[0100] like Figure 1-2 As shown, the method for feature extraction and matching of satellite video images provided by the present invention comprises the following steps:

[0101] Step S1: determining a satellite video image to be matched, and extracting corner points from the satellite video image to be matched;

[0102] Step S2: determining the difference between the gray value of each corner point and the gray value of the extreme point corresponding to the corner point, comparing the difference with a preset threshold, and performing a first filtering on the corner points; for each corner point after the first filtering, determining the stability of the corner point; and performing a second filtering on the corner points based on the stability;

[0103] Step S3: Based on the corner points after secondary filtering, characterize the satellite video image to be matched, and obtain a feature vector characterizing the satellite video image to be matched;

[0104] Step S4: Matching the feature vectors of the satellite video images to be matched.

[0105] The step S1: determining a satellite video image to be matched, and extracting corner points from the satellite video image to be matched, comprises:

[0106] Step S101: Divide the satellite video image to be matched into a plurality of image blocks, and extract a plurality of feature points from each image block, wherein:

[0107] The satellite video image of size w×h is divided into image blocks of m rows and n columns. A preset number of feature points are obtained from each image block by parallel processing. For any feature point P(x, y) extracted from the image block of the i-th row and j-th column, its real coordinates are calculated according to the following formula:

[0108]

[0109] In this embodiment, a strategy of extracting the entire image in blocks is adopted, which has the advantages of extracting a specified number of feature points from each block, which can ensure the uniformity of feature point distribution to a certain extent; and the feature extraction process can be accelerated by parallel acceleration.

[0110] Step S102: constructing a Gaussian-Laplacian operator, and using the Gaussian-Laplacian operator to extract a predetermined number of corner points from each image block, wherein:

[0111] The Gaussian-Laplacian operator is expressed as

[0112]

[0113] Among them, LoG (Laplacian of Gaussian) is the Gaussian-Laplacian operator, which describes the two-dimensional isotropic measure of the second-order spatial derivative of the image, ΔG σ (x, y) is the second-order derivative of the Gaussian convolution function here, x is the image pixel column coordinate, y is the image pixel row coordinate, G σ (x, y) is the Gaussian convolution function, and σ is the standard deviation of the Gaussian filter function.

[0114] The Gaussian-Laplace operator can be approximated by using the Gaussian function difference to obtain the approximate formula of the Gaussian-Laplace function as follows:

[0115]

[0116] Among them, k is 1.6, which is the scaling ratio of the standard deviation of the Gaussian distribution. The purpose is to obtain different Gaussian filter parameters by scaling with k, so as to obtain the Gaussian difference formed by subtracting the images filtered with different parameters; G(x, y, kσ) and G(x, y, σ) are the Gaussian filtering results with parameters σ and kσ respectively.

[0117] Use the Gaussian-Laplacian operator to extract a predetermined number of corner points from each image block that has been segmented, calculate and record the image block I ij Each of the (x, y) that achieves the maximum value in formula 4 is:

[0118]

[0119] Among them, f(x,y) is the gray value of point (x,y); the obtained (x,y) constitutes the corner points initially extracted from the satellite video image.

[0120] The advantage of approximation is that it can reduce the amount of calculation and improve the efficiency of the algorithm.

[0121] In this embodiment, parallel calculations are performed for each block image to extract image features respectively. Since the image is composed of discrete pixel grayscale values, mathematically, the image is a two-dimensional discrete domain. In the image, the edge area can be regarded as the place where the first-order derivative is larger in the image, and the feature point is the place where the second-order derivative is larger. That is, the place where there is an edge in both directions, and the intersection of the edge and the edge is also the corner point. A Gaussian kernel function is added to the edge detection operator. Such an operator is a high-pass filter, which is the sum of the second-order partial derivatives of the image grayscale function in two perpendicular directions.

[0122] In this embodiment, due to the high frame rate of the video image and the large amount of data in the matched image frame, it is necessary to further improve the speed while ensuring the uniformity of the feature points. The image is extracted by block extraction strategy and parallel calculation is used to improve the calculation efficiency. In view of the small scale change between satellite video images, the Gaussian-Laplace operator is used for preliminary feature extraction, and the approximation algorithm of the operator is used to reduce the amount of calculation and further improve the calculation speed.

[0123] The corner points obtained in step S1 may have errors, and generally include low-contrast feature points and unstable edge response points. According to the characteristics of the above two types of pseudo corner points, screening strategies are designed to remove such pseudo corner points.

[0124] The step S2 comprises:

[0125] Step S201: determine and filter low-contrast corner points, that is, remove candidate points that are sensitive to noise:

[0126] Let the selected corner point p0(x,y) and its corresponding extreme point The offset is defined as Δp(Δx, Δy). To illustrate the processing of grayscale values, let the grayscale value of any point p(x, y) be D(p). Taylor expansion of D(p) yields:

[0127]

[0128] Formula 5 is the matrix form of Taylor expansion. It is a local extreme point in the vicinity of point p0, so we take the derivative of formula 5 and set the derivative to 0, and we have:

[0129]

[0130] Substitute the obtained Δp into the Taylor expansion of D(p):

[0131]

[0132] Calculate the absolute value of the difference between the extreme point and the point to be selected, Take the gray value threshold Th=0.03, if Diff≤Th, then keep the candidate corner point; otherwise, the candidate corner point is considered unstable and filtered.

[0133] Step S202: Determine and delete unstable edge response points from the filtered candidate corner points, that is, perform the following operations on each of the filtered candidate corner points:

[0134] For the filtered candidate corner point p′(x,y), construct the Hessian matrix H of the filtered candidate corner point

[0135]

[0136] Among them, D xx , D xy , D yy It is the difference result of the neighborhood position of the corner point p′(x,y) to be selected.

[0137] Let α be the maximum eigenvalue of the H matrix, β be the minimum eigenvalue, and α and β can represent the gradients in the x and y directions respectively, then:

[0138] Tr(H)=D xx +D yy =α+β (Formula 9)

[0139] Det(H)=D xx D yy -(D xy ) 2 =αβ (Formula 10)

[0140] Let α=rβ, then:

[0141]

[0142] According to mathematical knowledge, (r+1) 2 / r has a value between two eigenvalues ​​(D xx , D yy ) are equal, and increase with the increase of r.

[0143] Therefore, the threshold r is set to 12:

[0144]

[0145] If the candidate corner point p′(x, y) satisfies formula 12, the candidate corner point is an unstable edge response point and is deleted; otherwise, the candidate corner point is retained.

[0146] In this embodiment, the principal curvature value is relatively large in the edge gradient direction, while the principal curvature value along the edge direction is relatively small. The unstable edge response points are removed by using the characteristic that the principal curvature of the DoG function D(p′) constructed by the candidate feature points is proportional to the eigenvalue of the Hessian matrix.

[0147] The step S3: characterizing the satellite video image to be matched based on the corner points after secondary filtering, and obtaining a feature vector characterizing the satellite video image to be matched, includes:

[0148] Filter to get the corner point set, Point={p1,…,p i ,…p num}, where 1≤i≤num, num is the number of corner points in the filtered corner point set;

[0149] For the corner point p i , select the corner point p i The surrounding 128 pairs of pixels inum1 ,Q inum1},S inum1 , Q inum1 is the corner point p i The surrounding pixels, and S inum1 , Q inum1 The distribution law conforms to the Gaussian distribution; each pixel pair uses a binary descriptor R inum1 To characterize,

[0150] Among them, 1≤num1≤128, then the corner point p i It can be represented by a 128-dimensional feature vector composed of 0 and 1; at this time, the feature vector of the satellite video image to be matched is obtained, and the feature vector is 128-dimensional, consisting of the corner point p i It is composed of the size relationship between the surrounding 128 groups of pixel pairs.

[0151] In this embodiment, since the inter-frame geometry and grayscale changes of satellite video in a short period of time are small, binary descriptors can be used directly for description. Compared with other feature description methods, binary expression storage and comparison are very fast. Although the descriptor scale and rotation adaptability are weak, since the inter-frame differences of satellite video are small, this method can meet the needs while maintaining speed. The vector of the binary descriptor consists of multiple 0s and 1s.

[0152] Step S4: Matching the feature vectors of the satellite video image to be matched, wherein:

[0153] The satellite video images to be matched are all represented by binary vectors of equal length. The binary vectors are used as binary strings to treat the satellite video images I a ,I b The binary strings corresponding to the feature points in the Calculate Hamming distances between each pair, that is

[0154]

[0155] Among them, s represents the sth value of the feature vector, dist h Represents the Hamming distance, dist s Calculated as follows:

[0156]

[0157] Among them, a s and b s I a and I b The value of the feature vector corresponding to the position of the feature point in .

[0158] When the Hamming distance dist h When it is greater than 0.8 times the vector length, it is considered that the satellite video image to be matched is I a ,I b This set of feature points in matches; otherwise, they do not match.

[0159] In this embodiment, since binary descriptors are used for feature description, the matching problem is equivalent to measuring the similarity between two binary strings, so the Hamming distance is selected as the metric function. The Hamming distance represents the sum of the number of different characters in corresponding positions in two equal-length strings.

[0160] In order to verify the feasibility of the method, discrete frames from a certain satellite video are selected for experimental testing. The selected satellite video size is 4096×3072, the surface resolution is 2m, and it covers approximately 8km×

[0161] 6km range. The landforms mainly include coasts, ports and urban artificial buildings, with dense textures and uniform distribution.

[0162] like Figure 1-2 As shown, the present invention discloses a fast feature extraction and matching method suitable for satellite video, comprising the following steps:

[0163] Step 1: Feature corner extraction

[0164] In order to speed up the corner point extraction process, the to-be-matched video image i (i=1, 2) is evenly divided into 4 blocks for processing according to the number of image pixels.

[0165] Each block of the two frames of images is convolved with the Gaussian-Laplacian operator shown in the following formula, and then the maximum value of the convolution result is obtained to obtain the candidate corner points of each block of the image. All the candidate corner points of the four blocks of images are the candidate corner points of the entire image.

[0166] Δ[G σ (x i ,y i )*f(x i ,y i )])=(LoG*f(x i ,y i ))

[0167] (i=1, 2)

[0168] The corner points to be selected from some extracted video images are as follows: Figure 3 shown.

[0169] Step 2: After obtaining the selected corner points of the video image i (i=1, 2), the low-contrast corner points and edge response points are filtered respectively to obtain relatively stable and reliable feature corner points of the video image i (i=1, 2).

[0170] Remove low-contrast corners.

[0171] Suppose there is a corner point x(x,y) to be selected, its offset is defined as Δx(Δx,Δy), and the grayscale contrast is the absolute value of D(x) |D(x)|. Then applying Taylor expansion to D(x) at point x gives:

[0172]

[0173] Substitute the obtained Δx into the Taylor expansion of D(x):

[0174]

[0175] is the extreme point of D(x).

[0176] Let the contrast threshold be T, so if Then keep the feature point, otherwise delete it.

[0177] 2. Remove unstable edge response points

[0178] The Hessian matrix H is constructed as follows.

[0179]

[0180] Among them, D xx , D xy , D yy is the difference of the candidate point's neighborhood position.

[0181] Let α be the largest eigenvalue of the H matrix and β be the smallest eigenvalue,

[0182] α and β represent the gradients in the x and y directions.

[0183] Let α=rβ, then:

[0184]

[0185] Set the threshold r in the above formula and construct the following:

[0186]

[0187] If the above formula is established, the feature point is removed, otherwise it is retained.

[0188] After removing the pseudo corners in two ways, the feature points of video image 1 are as follows: Figure 4 shown.

[0189] Step 3: After obtaining stable corner points of the video image i (i=1, 2) respectively, a binary descriptor is used to describe the extracted corner points, thereby obtaining a feature description vector of the video image i (i=1, 2).

[0190] Select 128 pairs of pixels p around the feature point k and q k (k=1,2…128),p k and q k Generally, the selection is made according to a certain distribution law. In this case, Gaussian distribution is used. The grayscale of each pair of pixels is compared according to the following formula.

[0191]

[0192] 128 comparison results are obtained as a description of the feature point, and a 128-dimensional feature vector composed of 0 and 1 can be obtained.

[0193] Step 4: Match the feature points obtained in different frames to obtain matching feature point pairs, such as Figure 5 shown.

[0194] Since each feature point of the video image i (i=1, 2) is described by a binary descriptor, the matching problem is equivalent to measuring the similarity between two binary strings, and the Hamming distance is selected as the metric function.

[0195] For two feature point descriptions a and b with a length of 128, the Hamming distance is expressed as follows:

[0196] (h = 1, 2...n, n is the number of eigenvectors)

[0197] where dist k Calculated as follows:

[0198]

[0199] Compared with the SIFT algorithm, this algorithm has less computational effort and higher efficiency in feature description and matching; compared with the ORB algorithm, this method has better effects in corner detection and filtering, and the extracted feature points are more reliable.

[0200] The present invention also provides a device for extracting and matching features of satellite video images, such as Figure 6 As shown, the device includes a corner point acquisition module: configured to determine a satellite video image to be matched, and extract corner points from the satellite video image to be matched;

[0201] Corner point filtering module: configured to determine the difference between the gray value of each corner point and the gray value of the extreme point corresponding to the corner point, compare the difference with a preset threshold, and perform a first filtering on the corner points; for each corner point after the first filtering, determine the stability of the corner point; and perform a second filtering on the corner points based on the stability;

[0202] A feature vector acquisition module: configured to characterize the satellite video image to be matched based on the corner points after secondary filtering, and obtain a feature vector characterizing the satellite video image to be matched;

[0203] Matching module: configured to match feature vectors of satellite video images to be matched.

[0204] The above specific embodiments only describe the design principle of the present invention. The shapes and names of the components in the description may be different and are not limited. Therefore, those skilled in the art in the field of the present invention may modify or replace the technical solutions recorded in the above embodiments; and these modifications and replacements do not deviate from the creative purpose and technical solutions of the present invention and should all fall within the protection scope of the present invention.

Claims

1. A method for feature extraction and matching of satellite video images, characterized in that: include: Step S1: determining a satellite video image to be matched, and extracting corner points from the satellite video image to be matched; Step S2: determining the difference between the grayscale value of each corner point and the grayscale value of the extreme point corresponding to the corner point, comparing the difference with a preset threshold, and performing a first filtering on the corner points; for each corner point after the first filtering, determining the stability of the corner point; Based on stability, the corner points are filtered twice; Step S3: Based on the corner points after secondary filtering, characterize the satellite video image to be matched, and obtain a feature vector characterizing the satellite video image to be matched; Step S4: matching the feature vectors of the satellite video images to be matched; The step S1 specifically includes: Step S101: Divide the satellite video image to be matched into a plurality of image blocks I ij , extract several feature points from each image block, image block I ij Represents the image block in row i and column j; Step S102: constructing a Gaussian-Laplacian operator, and using the Gaussian-Laplacian operator to extract a predetermined number of corner points from each image block, wherein: The Gaussian-Laplacian operator is Among them, LoG (Laplacian of Gaussian) is the Gaussian-Laplacian operator, which describes the two-dimensional isotropic measure of the second-order spatial derivative of the image, ΔG σ (x, y) is the second-order derivative of the Gaussian convolution function at this point, x is the image pixel column coordinate, y is the image pixel row coordinate, G σ (x, y) is the Gaussian convolution function, σ is the standard deviation of the Gaussian filter function; Where k is 1.6, G(x, y, kσ) and G(x, y, σ) are Gaussian filtering results with parameters σ and kσ respectively; Use the Gaussian-Laplacian operator to extract a predetermined number of corner points from each image block that has been segmented, calculate and record the image block I ij Each (x, y) that maximizes Formula 4 is: Among them, f(x,y) is the gray value of point (x,y); each (x,y) obtained constitutes the corner points initially extracted from the satellite video image; The step S2 comprises: Step S201: determine and filter low-contrast corner points, that is, remove candidate points that are sensitive to noise: For each candidate corner point, let the candidate corner point be p0(x,y), and the corresponding extreme point The offset is defined as Δp(Δx, Δy), and the absolute value of the difference between the extreme point and the corner point to be selected is calculated. Take the gray value threshold Th=0.03, if Diff≤Th, then keep the candidate corner point; otherwise, consider that the candidate corner point is unstable and filter the candidate corner point; The extreme point The gray value of , D(p0) is the gray value of the corner point p0 to be selected; Step S202: Determine and delete unstable edge response points from the filtered candidate corner points, that is, perform the following operations on each of the filtered candidate corner points: For the filtered candidate corner point p′(x,y), construct the Hessian matrix H of the filtered candidate corner point Among them, D xx , D xy , D yy is the difference result of the filtered candidate contact point p′(x,y) field position; Let α be the maximum eigenvalue of the H matrix, β be the minimum eigenvalue, and α and β can represent the gradients in the x and y directions respectively, then: Tr(H)=D xx +D yy =α+β Det(H)=D xx D yy -(D xy ) 2 =αβ Set the threshold r, r value is 12: If the candidate corner point p′(x, y) meets the judgment condition, the candidate corner point is an unstable edge response point and is deleted; otherwise, the candidate corner point is retained.

2. The method according to claim 1, characterized in that The step S3 comprises: Filter to get the corner point set Point = {p1,…,p i ,…p num }, where 1≤i≤num, num is the number of corner points in the filtered corner point set; For the corner point p i , select the corner point p i The surrounding 128 pairs of pixels inum1 ,Q inum1 },S inum1 , Q inum1 is the corner point p i The surrounding pixels, and S inum1 , Q inum1 The distribution law conforms to the Gaussian distribution; each pixel pair uses a binary descriptor R inum1 To characterize, Among them, 1≤num1≤128, then the corner point p i It can be represented by a 128-dimensional feature vector composed of 0 and 1; at this time, the feature vector of the satellite video image to be matched is obtained, which is composed of the corner point p i It is composed of the size relationship between the surrounding 128 groups of pixel pairs.

3. The method according to claim 2, characterized in that Step S4: Matching the feature vectors of the satellite video image to be matched, wherein: The satellite video images to be matched are all represented by binary vectors of equal length. The binary vectors are used as binary strings to treat the satellite video images I a ,I b The binary strings corresponding to the feature points in the Calculate Hamming distances between each pair, that is Among them, s represents the sth value of the feature vector, dist h Represents the Hamming distance, dist s Calculated as follows: Among them, a s and b s I a and I b The value of the corresponding position of the feature vector of the feature point in ; When the Hamming distance dist h When it is greater than 0.8 times the vector length, it is considered that the satellite video image to be matched is I a ,I b This set of feature points in matches; otherwise, they do not match.

4. A device for feature extraction and matching of satellite video images, characterized in that: The device comprises: Corner point acquisition module: configured to determine the satellite video image to be matched, and extract corner points from the satellite video image to be matched; Corner point filtering module: configured to determine the difference between the gray value of each corner point and the gray value of the extreme point corresponding to the corner point, compare the difference with a preset threshold, and perform a first filtering on the corner points; for each corner point after the first filtering, determine the stability of the corner point; and perform a second filtering on the corner points based on the stability; A feature vector acquisition module: configured to characterize the satellite video image to be matched based on the corner points after secondary filtering, and obtain a feature vector characterizing the satellite video image to be matched; Matching module: configured to match feature vectors of satellite video images to be matched; The corner point acquisition module comprises: Image segmentation submodule: configured to divide the satellite video image to be matched into a plurality of image blocks I ij , extract several feature points from each image block, image block I ij Represents the image block in row i and column j; Corner point acquisition submodule: configured to construct a Gaussian-Laplacian operator, and use the Gaussian-Laplacian operator to extract a predetermined number of corner points from each image block, wherein: The Gaussian-Laplacian operator is Among them, LoG (Laplacian of Gaussian) is the Gaussian-Laplacian operator, which describes the two-dimensional isotropic measure of the second-order spatial derivative of the image, ΔG σ (x, y) is the second-order derivative of the Gaussian convolution function at this point, x is the image pixel column coordinate, y is the image pixel row coordinate, G σ (x, y) is the Gaussian convolution function, σ is the standard deviation of the Gaussian filter function; Where k is 1.6, G(x, y, kσ) and G(x, y, σ) are Gaussian filtering results with parameters σ and kσ respectively; Use the Gaussian-Laplacian operator to extract a predetermined number of corner points from each image block that has been segmented, calculate and record the image block I ij Each (x, y) that maximizes Formula 4 is: Among them, f(x,y) is the gray value of point (x,y); each (x,y) obtained constitutes the corner points initially extracted from the satellite video image; The corner point filtering module comprises: The first filtering submodule is configured to determine and filter low-contrast corner points, that is, to remove candidate points that are sensitive to noise: For each candidate corner point, let the candidate corner point be p0(x,y), and the corresponding extreme point The offset is defined as Δp(Δx, Δy), and the absolute value of the difference between the extreme point and the corner point to be selected is calculated. Take the gray value threshold Th=0.03, if Diff≤Th, then keep the candidate corner point; otherwise, consider that the candidate corner point is unstable and filter the candidate corner point; The extreme point The gray value of , D(p0) is the gray value of the corner point p0 to be selected; The second filtering submodule is configured to determine and delete unstable edge response points from the filtered candidate corner points, that is, for each of the filtered candidate corner points, perform the following operations: For the filtered candidate corner point p′(x,y), construct the Hessian matrix H of the filtered candidate corner point Among them, D xx , D xy , D yy is the difference result of the filtered candidate contact point p′(x,y) field position; Let α be the maximum eigenvalue of the H matrix, β be the minimum eigenvalue, and α and β can represent the gradients in the x and y directions respectively, then: Tr(H)=D xx +D yy =α+β Det(H)=D xx D yy -(D xy ) 2 =αβ Set the threshold r, where r is 12: If the candidate corner point p′(x, y) meets the judgment condition, the candidate corner point is an unstable edge response point and is deleted; otherwise, the candidate corner point is retained.

5. The device according to claim 4, characterized in that The feature vector acquisition module comprises: Corner point set acquisition submodule: configured to filter and obtain the corner point set Point = {p1, ..., p i ,…p num }, where 1≤i≤num, num is the number of corner points in the filtered corner point set; Feature acquisition submodule: configured for corner point p i , select the corner point p i The surrounding 128 pairs of pixels inum1 ,Q inum1 },S inum1 , Q inum1 is the corner point p i The surrounding pixels, and S inum1 , Q inum1 The distribution law conforms to the Gaussian distribution; each pixel pair uses a binary descriptor R inum1 To characterize, Among them, 1≤num1≤128, then the corner point p i It can be represented by a 128-dimensional feature vector composed of 0 and 1; at this time, the feature vector of the satellite video image to be matched is obtained, which is composed of the corner point p i It is composed of the size relationship between the surrounding 128 groups of pixel pairs.

6. The device according to claim 5, characterized in that The matching module, wherein: The satellite video images to be matched are all represented by binary vectors of equal length. The binary vectors are used as binary strings to treat the satellite video images I a ,I b The binary strings corresponding to the feature points in the Calculate Hamming distances between each pair, that is Among them, s represents the sth value of the feature vector, dist h Represents the Hamming distance, dist s Calculated as follows: Among them, a s and b s I a and I b The value of the corresponding position of the feature vector of the feature point in ; When the Hamming distance dist h When it is greater than 0.8 times the vector length, it is considered that the satellite video image to be matched is I a ,I b This set of feature points in matches; otherwise, they do not match.

Images