Remote Sensing Image Target Detection Method and System

Abstract

The present invention provides a method and system for remote sensing image target detection. The method includes a target recognition and extraction step and a false alarm elimination step. According to the characteristics of on-orbit data processing, the present invention utilizes the features of typical large-scale ground targets such as airports, bridges, and the edges, shapes, and scale invariance of oil depots, designs corresponding feature description, detection, and matching algorithms, and optimizes them to achieve real-time and accurate detection and processing of large-scale ground targets on orbit. By applying the remote sensing data processing method in the present invention, fixed targets can be quickly and accurately detected and recognized from high-resolution remote sensing images; at the same time, by establishing a multi-feature matching model as the classification basis for fixed targets, the accuracy of target recognition can be effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, especially the on-board remote sensing data processing technology. Background Art

[0002] The problem of object detection has always been an important research direction in computer vision. Its main task is to determine whether there is an object to be detected in an image, and extract and accurately locate it. To solve this problem, methods in mathematics, physics, information science, etc. are comprehensively used to realize the inversion from ground object data records to ground object features and their spatio-temporal distribution and changes, which involves many aspects such as remote sensing, pattern recognition, and machine vision, and is a typical interdisciplinary problem.

[0003] With the development of sensor technology, space technology, information processing technology, small satellite technology, etc., the spatio-temporal and spectral resolution capabilities of remote sensing images have been significantly improved, and the on-board remote sensing data real-time processing technology has also developed rapidly. On-board object real-time detection is one of the current research hotspots. In modern warfare, how to ensure the real-time, accurate, and flexible remote sensing effective information is one of the main problems faced by the current application of remote sensing data. At present, the on-board processing ability of existing remote sensing satellites in China is weak, and the work of remote sensing image object recognition and other aspects mainly relies on ground terminals.

[0004] The existing remote sensing image object detection methods can be divided into two types: one is bottom-up data-driven, and the other is top-down knowledge-driven. The former, regardless of the type of the recognized object, first performs underlying processing such as general segmentation, labeling, and feature extraction on the original image, and then matches the feature vectors of each segmented and labeled region (object) with the object model. This type of method is usually based on classification technology. The latter, according to the description model of the recognized object, first makes hypotheses about the possible features in the image, and then performs segmentation, labeling, and feature extraction purposefully according to the hypotheses, and performs precise matching with the object on this basis. This type of method usually requires certain prior knowledge and is based on knowledge representation and reasoning technology.

[0005] How to ensure the real-time, accurate, and flexible remote sensing effective information is one of the main problems faced by the current application of remote sensing data. Summary of the Invention

[0006] The present invention aims to solve at least one of the technical problems existing in the prior art or related technologies.

[0007] To solve the above problems, the present invention provides a remote sensing image object detection method and system.

[0008] According to the first aspect of the present invention, a remote sensing image object detection method is provided, including an object recognition and extraction step and a false alarm elimination step:

[0009] The target recognition and extraction steps include: preprocessing the search image and the target image; traversing the images and extracting feature vectors, matching the feature vectors of the search image with those of the target image, and if the matching result is positive, recording the target position, otherwise not; after completing the traversal and matching of the entire search image, performing the false alarm elimination step;

[0010] The false alarm elimination step includes: using a multi-feature matching model for false alarm elimination. The basis for establishing the multi-feature matching model as the classification of fixed targets is:

[0011]

[0012] where T is the output of the multi-feature matching model, which serves as the classification threshold, X i is a selected single feature value, a i is the weight of each feature value in the entire matching model, n represents the number of image features; i ranges from 0 to n, and n is greater than or equal to 2.

[0013] Furthermore, the image features in the false alarm elimination step are: grayscale feature, length feature, width feature, aspect ratio feature, and mass density feature.

[0014] Furthermore, the recognized targets include land targets and sea targets.

[0015] Furthermore, when the target is a land target, the preprocessing of the search image and the target image includes: performing Gaussian filtering on the search image and the target image to suppress noise and performing downsampling.

[0016] Furthermore, when the target is a land target, the specific matching of the features of the search image and the target image includes: continuously filtering and downsampling the input search image and target image through Gaussian kernel functions of different scales to form Gaussian pyramid images, and extracting image feature points for matching.

[0017] Furthermore, the matching of the features of the search image and the target image further includes: comparing each feature point in the pyramid scale space with the feature points at adjacent scales and adjacent positions one by one to obtain the local extreme positions for matching.

[0018] Furthermore, the matching of the features of the search image and the target image further includes: generating feature vectors based on the information of the local extreme positions for matching.

[0019] Further, when the target is a maritime target, the steps of extracting image feature points from the search image and the target image include: detecting the target using the phase congruency method, and the detection includes: straight line feature extraction, circular feature extraction, and / or conical feature extraction.

[0020] Further, the phase congruency method includes:

[0021] Performing Fourier transform on the search image and the target image, respectively obtaining the amplitude and phase angle; taking the logarithm of the amplitude, obtaining the result within a certain frequency range after filtering, and then performing inverse transform together with the phase angle.

[0022] According to the second aspect of the present invention, there is provided a remote sensing image target detection system, including a target recognition and extraction module and a false alarm elimination module. The target recognition and extraction module performs the target recognition and extraction steps of any one of the above, and the false alarm elimination module performs the false alarm elimination steps of any one of the above.

[0023] According to the characteristics of on-orbit data processing, the present invention designs corresponding feature description, detection, and matching algorithms by using the characteristics of typical large-scale ground targets such as airports, bridges, and the edges, shapes, and scale invariance of oil depots, and optimizes them, realizing real-time and accurate detection and processing of large-scale ground targets on orbit. By applying the remote sensing data processing method of the present invention, fixed targets can be detected and recognized quickly and accurately from high-resolution remote sensing images; at the same time, by establishing a multi-feature matching model as the classification basis for fixed targets, the accuracy of target recognition can be effectively improved.

[0024] At the same time, the method and device of the present invention can extract feature points from the target image, and can use the method of multi-scale direction gradient histogram matching to extract the corresponding feature points in the target reference image and the image to be detected. By establishing the corresponding relationship between the registration primitives, the transformation model parameters are solved to realize the automatic registration of the target. To improve the algorithm efficiency.

[0025] The additional aspects and advantages of the present invention will be given in the following description part, some will become obvious from the following description, or will be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and / or additional aspects and advantages of the present invention will become obvious and easy to understand from the description of the embodiments in conjunction with the following drawings, wherein:

[0027] Figure 1 is a flowchart of the remote sensing data target detection method according to the present invention;

[0028] Figure 2 is a flowchart of the false alarm elimination step according to the present invention;

[0029] Figure 3 It is a structural diagram of a remote sensing data target detection system according to the present invention. Specific embodiments

[0030] In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0031] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0032] The present invention provides a method for detecting fixed targets in remote sensing images based on multi-scale features, which can effectively detect fixed targets in remote sensing images, such as ships, airplanes, oil storage tanks, ports, bridges, etc.

[0033] The following will describe the method steps of the present application in detail.

[0034] Refer to Figure 1 , the process method of the remote sensing data target detection method includes a target recognition and extraction step and a false alarm elimination step executed by a false alarm elimination module, specifically:

[0035] (1) Image preprocessing.

[0036] Perform Gaussian filtering on the input target reference image I 1 and the image to be searched to effectively suppress the noise points in the image.

[0037] The Gaussian filtering function is:

[0038] Obtain the preprocessed target reference image I 1 and the image to be searched I 2

[0039] (2) Image downsampling.

[0040] Set the downsampling ratio and sampling parameters, and these parameters can be reasonably set according to actual needs, K 1 , K 2 , and the downsampling ratio can also be reasonably set according to actual needs. For example, the default value of the target image is 2, and the default value of the image to be searched is 4, and thumbnails of the corresponding images are generated to facilitate the traversal of the active window through the image.

[0041] (3) Feature point detection.

[0042] The detection of feature points is divided into the detection of ground targets and the detection of maritime targets.

[0043] For the detection of ground targets, since the shapes of the targets to be detected are fixed, it is necessary to first extract the features of the detection targets, and use the feature points to match the targets on the satellite. The sift algorithm or the fast improved algorithm surf of sift can be used. Since sift uses a multi-layer decomposition method, the algorithm is time-consuming, which is an inherent defect of the sift algorithm. The present invention uses the fast improved algorithm surf algorithm for sift to optimize the algorithm complexity design to meet the near-real-time requirement.

[0044] Generation of scale space.

[0045] The purpose of the scale space theory is to simulate the multi-scale features of image data. The Gaussian convolution kernel is the only linear kernel for implementing scale transformation. Therefore, the scale space of a two-dimensional image is defined as:

[0046] L(x, y, σ) = G(x, y, σ) * I(x, y) where G(x, y, σ) is a scale-variable Gaussian function: (x, y) are spatial coordinates, and σ is the scale coordinate.

[0047] The input image is continuously filtered and downsampled by Gaussian kernel functions of different scales to form a Gaussian pyramid image, and then the difference between two Gaussian images of adjacent scales is obtained to get the pyramid multi-scale space. The information generated by the scale space is used for feature point detection.

[0048] The steps for detecting the key points (spatial extreme points or feature points) are specifically as follows:

[0049] Each point in the pyramid scale space is compared with the points of adjacent scales and adjacent positions one by one, and the obtained local extreme position is the position and corresponding scale where the key points are located.

[0050] The key point direction assignment samples in the neighborhood window centered on the key point, and uses a histogram to statistically analyze the gradient directions of the neighborhood pixels. The range of the gradient histogram is 0 to 360 degrees, with one bin every 10 degrees, for a total of 36 bins.

[0051] For the detection of maritime targets such as ships, the phase consistency method can be used:

[0052] a. Perform Fourier transform on the above image to obtain the amplitude and phase angle respectively;

[0053] b. Take the logarithm of the amplitude, and obtain the result within a certain frequency range after filtering;

[0054] c. Perform inverse transformation on the said result and the phase angle together.

[0055] This method utilizes the principle of homomorphic filtering, which can highlight heterogeneous regions under certain background conditions, overcome the stripe noise interference in infrared images to the greatest extent, and achieve accurate detection of ship targets; finally, a feature vector is generated.

[0056] (4) Generate a feature vector, and the specific steps are as follows:

[0057] To ensure the rotation invariance of the feature vector, it is necessary to rotate the position and direction of the image gradient in the neighborhood (mσ(Bp + 1)√2 x mσ(Bp + 1)√2) near the feature point by an angle θ around the feature point, that is, rotate the x-axis of the original image to the same direction as the main direction. The rotation formula is as follows.

[0058]

[0059] After rotating the position and direction of the image gradient in the neighborhood near the feature point, take an image region of size mσBp x mσBp centered on the feature point in the rotated image. And divide it into Bp X Bp sub-regions at equal intervals, with each interval being mσ pixels.

[0060] Calculate the gradient direction histogram in 8 directions within each sub-region, plot the cumulative value of each gradient direction to form a seed point. Different from finding the main direction of the feature point, at this time, the gradient direction histogram of each sub-region divides 0° - 360° into 8 direction ranges, each range being 45°. In this way, each seed point has gradient intensity information in 8 directions. Since there are 4X4 (Bp X Bp) sub-regions, there are a total of 4X4X8 = 128 data, and finally a 128-dimensional sift feature vector is formed. Similarly, Gaussian weighting needs to be performed on the feature vector. The weighting uses a standard Gaussian function with a variance of mσBp / 2, where the distance is the distance of each point relative to the feature point. Using Gaussian weights is to prevent a small change in position from bringing a large change to the feature vector and to assign a smaller weight to points far from the feature point to prevent incorrect matching. Obtain the feature vector of the target image and the feature vector of the window image through the feature vector generation step.

[0061] (5) Feature vector matching, and the specific steps are as follows:

[0062] The LBP features of an image are related to the number of pixels, but the SIFT features are not as numerous. Therefore, when fusing, the first step is to align the SIFT and LBP features at the same position. The position information of the SIFT features is stored in the 'pt' of the KeyPoints structure generated in the detect method. After aligning to the same position points, methods such as direct addition and matrix multiplication can be used to fuse the two features. Additionally, we need to remove those extreme points at the edge positions. Some extreme points are located at the edge positions of the image. Since it is difficult to locate the edge points of the image and they are also easily affected by noise, we consider these points as unstable extreme points and need to remove them to improve the stability of the key points.

[0063] When the feature vectors can be matched, the target position will be recorded. When the feature vectors cannot be matched, the target position will not be recorded, and then it is checked whether the traversal is completed. If so, it enters the false alarm elimination module. If not, it returns to the sliding window to traverse the image I. 2 Continue to loop this process.

[0064] In one embodiment, the extraction of the target image can specifically be to extract airport targets based on line features:

[0065] Under the condition of knowing the region shape in advance, the Hough transform can be used to conveniently obtain the boundary curve and connect the discontinuous pixel edge points.

[0066] In the X-Y coordinate axes, the points on the coordinate axes can all be represented by the general straight line equation:

[0067] y = ax + b

[0068] Among them, a and b are the meanings in the straight line equation and are constants respectively.

[0069] After the points in the image space are converted into straight lines in the parameter space, the points in the parameter space are accumulated: ρ = xcosθ + ysinθ.

[0070] In another embodiment, the extraction of the target image is to extract oil depot targets based on circle features:

[0071] Using the hough transform to detect and determine a circle in the image domain (X-Y plane), first, it is necessary to assume a set of points on the determined circle. (x, y) is a point in the set, and its equation in the parameter space (a, b, r) is: (x - a) 2 +(y - b) 2 = r 2Where a and b are the coordinates of the center point, and r is the radius. If the unknowns in the above formula are reversed such that x and y are constants while a, b, and r are unknowns, then the formula after reversing the unknowns corresponds to the equation of a cone. Any arbitrarily determined point on the circle corresponds to a three-dimensional conical surface in the parameter space.

[0072] After completing the traversal of all images, false alarms of suspected targets are eliminated in the original image.

[0073] Exemplarily, using the above method to extract ships will wrongly extract islands as ship targets. The correct extraction of high-resolution ship targets on the sea cannot be achieved through a single grayscale feature. Each feature only represents part of the features of a certain type of thing. It is unreasonable to use any single feature as the classification feature of a certain type of thing. After a large number of experiments, the present invention uses five features, namely grayscale, length, width, aspect ratio, and mass density, to jointly describe fixed targets in order to fully distinguish them from other fixed targets.

[0074] Based on the above characteristics, a multi-feature matching model is established as the classification basis for fixed targets:

[0075]

[0076] Where T is the output of the model and serves as the classification threshold, X i is the selected single feature value, a i is the weight of each feature value in the entire model, and n is the number of features. In the experiment, n = 0, 1, 2, 3, 4, a 0 = 0.4, a 1 = 0.2, a 2 = 0.2, a 3 = 0.1, a 4 = 0.1 The feature values X 0 , X 1 , X2, X 3 , X 4 are respectively the grayscale feature, length feature, width feature, aspect ratio feature, and mass density feature of the target area.

[0077] Figure 2 Shows the steps of false alarm elimination according to the present invention, including:

[0078] Input the target position data set generated by the method of the above reference Figure 1 into the above multi-feature matching model, perform matching detection on the target feature vector and the window feature vector. If the matching result meets the predetermined judgment condition, record the target position to form a target position data set; if not, do not record the target position, thereby achieving false alarm elimination. After completing the traversal of all target positions, a target position data set after false alarm elimination is obtained.

[0079] Reference Figure 3 , Figure 3 shows the structure of a remote sensing data target detection system according to the present invention, including a target recognition and extraction module 1 and a false alarm elimination module 2. The target recognition and extraction module performs any one of the above target recognition and extraction steps, and the false alarm elimination module performs any one of the above false alarm elimination steps.

[0080] At the same time, it can also be known that the method of the present invention can be implemented by computer software or hardware. By storing corresponding instructions through a computing storage medium and executed by a processor, the method of the present invention can also be realized.

[0081] It can be understood that although the present invention has been disclosed above with preferred embodiments, the above embodiments are not intended to limit the present invention. For any person skilled in the art, without departing from the scope of the technical solution of the present invention, many possible changes and modifications can be made to the technical solution of the present invention by using the above-disclosed technical content, or modified into equivalent embodiments with equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for remote sensing image target detection, characterized in that, it includes a target recognition and extraction step and a false alarm elimination step: The target recognition and extraction step includes: preprocessing the image to be searched and the target image; traversing the images and extracting feature vectors, matching the feature vectors of the image to be searched with those of the target image, if the matching result is yes, record the target position, otherwise do not record; after completing the traversal and matching of the entire image to be searched, execute the false alarm elimination step; where, when the target is a maritime target, the step of extracting feature vectors from the image to be searched and the target image includes: using the phase consistency method to detect the target, and the detection includes: straight line feature extraction, circular feature extraction, and / or conical feature extraction; when performing matching, for the sift and lbp features corresponding to the same position, the position information of the sift feature is saved in the pt of the KeyPoints structure generated in the detect method, and after corresponding to the same position point, the sift and lbp features are fused using the direct addition and matrix multiplication methods; The false alarm elimination step includes: using a multi-feature matching model to eliminate false alarms, and the basis for establishing the multi-feature matching model as a classification of fixed targets is: where T is the output of the multi-feature matching model, which serves as the classification threshold, and X i is the selected single feature value, and a i is the weight of each feature value in the entire matching model, n represents the number of image features; i takes values from 0 to n, and n is greater than or equal to 2; among them, The image features in the false alarm elimination step are: gray level feature, length feature, width feature, aspect ratio feature, and mass density feature.

2. The method according to claim 1, characterized in that: the recognized targets include land targets and maritime targets.

3. The method according to claim 1, characterized in that: when the target is a land target, the preprocessing of the image to be searched and the target image includes: performing Gaussian filtering on the image to be searched and the target image to suppress noise, and performing downsampling.

4. The method according to claim 3, characterized in that, when the target is a land target, the specific matching of the features of the image to be searched and the target image includes: continuously filtering and downsampling the input image to be searched and the target image through Gaussian kernel functions of different scales to form Gaussian pyramid images, and extracting image feature points for matching.

5. The method according to claim 4, characterized in that: the matching of the features of the image to be searched and the target image further includes: comparing each feature point in the pyramid scale space with the feature points in the adjacent scale and adjacent positions one by one, and obtaining the local extreme positions for matching.

6. The method according to claim 5, characterized in that: the matching of the features of the image to be searched and the target image further includes: generating feature vectors based on the information of the local extreme positions for matching.

7. The method according to claim 1, characterized in that: the phase consistency method therein includes: performing Fourier transform on the image to be searched and the target image, respectively obtaining the amplitude and phase angle; taking the logarithm of the amplitude, obtaining the result within a certain frequency after filtering, and then performing inverse transform together with the phase angle.

8. A remote sensing image target detection system, characterized in that, It includes a target recognition and extraction module and a false alarm elimination module. The target recognition and extraction module performs the target recognition and extraction steps in any one of claims 1-7, and the false alarm elimination module performs the false alarm elimination steps in any one of claims 1-7.

Images