Ground feature recognition method and device based on aerial image, terminal equipment and storage medium

Abstract

The invention discloses a ground feature recognition method and device based on an aerial image, terminal equipment and a storage medium, and the method comprises the steps: obtaining aerial image data of a target city, firstly extracting ground feature features in the aerial image data, and obtaining corresponding target ground feature features; according to the method, the junction similar to the target ground feature feature is preliminarily screened out by performing difference comparison on the junction features of the adjacent regions, and then two target regions corresponding to the junction can be quickly positioned to perform feature matching, so that the regions meeting the requirements can be quickly screened out on the basis of an effective screening mechanism, and the accuracy of feature matching is improved. A large amount of computing resources do not need to be wasted in areas which do not conform to requirements obviously, and the processing speed of ground feature recognition is increased; even if the number of the areas is large, the method can also quickly screen and locate the corresponding target areas for feature comparison and generation of the recognition result, thereby improving the efficiency of ground feature recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of ground object recognition, and in particular to a ground object recognition method, device, terminal equipment and storage medium based on aerial images. Background Art

[0002] In modern smart cities, Geographic Information System (GIS) is an important component that can realize the efficient collection, processing and application of land objects within the city. Feature Recognition in Remote Sensing generally refers to the automatic or semi-automatic classification and identification of surface covers by using remote sensing technology, such as image data obtained by satellites and drones, through specific methods and algorithms. These surface covers are usually called "land features", which can include different types of natural and man-made objects, such as water bodies, vegetation, buildings, roads, etc.

[0003] In GIS systems, sensor platforms, data storage and analysis modules are key components. Among them, sensor platforms such as optical or infrared cameras can obtain images of target areas, while cloud computing and big data technologies can store and manage massive image data, and deep learning algorithms such as semantic segmentation and target detection can also be used to automatically identify and analyze aerial objects or geographic information about the city.

[0004] In the existing technology, when analyzing and processing objects in aerial images of cities, each area needs to be analyzed and compared in detail. However, due to the lack of an effective screening mechanism, a large amount of computing resources will be wasted on areas that obviously do not meet the requirements. When the number of areas is large, the processing speed of object recognition will be slow and inefficient. Summary of the invention

[0005] The embodiments of the present invention provide a method, apparatus, terminal device and storage medium for identifying objects based on aerial images. Based on an effective screening mechanism (such as a feature comparison mechanism at the junction), areas that meet the requirements are quickly screened out without wasting a large amount of computing resources on areas that obviously do not meet the requirements. The processing speed of object identification is improved, and the problem of a large amount of computing resources wasted and low efficiency of object identification in the prior art due to the lack of an effective screening mechanism can be effectively solved.

[0006] An embodiment of the present invention provides a method for identifying ground objects based on aerial images, comprising:

[0007] Acquire an aerial image of a target city; wherein the target city corresponds to a plurality of regional images; each regional image corresponds to a local feature; and a junction corresponding to two adjacent regional images corresponds to a junction feature;

[0008] Extracting the features of the ground objects in the aerial image to generate the features of the target ground objects;

[0009] Subtract the target feature from each boundary feature to generate a difference value corresponding to each boundary feature, and use the boundary feature corresponding to the difference value not greater than a preset difference threshold as the target boundary feature, and then use the adjacent area images corresponding to the target boundary feature as the first target area image and the second target area image;

[0010] Comparing the target ground feature with the sample ground feature corresponding to the first target area image and the second target area image respectively;

[0011] When it is determined that the target object feature matches the sample object feature corresponding to the first target area successfully, a recognition result is generated for indicating that the aerial image belongs to the first target area image, or,

[0012] When it is determined that the target object feature matches the sample object feature corresponding to the second target area successfully, a recognition result is generated to indicate that the aerial image belongs to the second target area image.

[0013] Preferably, the target ground feature includes: target edge feature, target area feature, target texture feature, target color feature and target shape feature;

[0014] The step of extracting the ground object features in the aerial image to generate target ground object features includes:

[0015] Extracting target area features, target texture features, target color features, and target shape features in the aerial image, and using edge lines in the aerial image as target edge features corresponding to the aerial image;

[0016] The target edge features, target area features, target texture features, target color features and target shape features are used as target ground object features corresponding to the aerial image.

[0017] Preferably, it also includes: an aerial image database; the aerial image database includes a number of sample ground feature and a number of boundary ground feature; each sample ground feature corresponds to each area image in the target city;

[0018] Each sample feature includes: a first sample edge feature, a first sample region feature, a first sample texture feature, a first sample color feature, and a first sample shape feature; each boundary feature includes: a second sample edge feature, a second sample region feature, a second sample texture feature, a second sample color feature, and a second sample shape feature;

[0019] The construction of the aerial image database includes:

[0020] Acquire an original aerial image corresponding to a target city; wherein the original aerial image contains images of all regions of the target city;

[0021] Perform preprocessing on each original image to generate a processed target image;

[0022] Calculate the gradient magnitude and gradient direction of each pixel in the target image. For each pixel, take the maximum gradient magnitude in the same gradient direction as the target gradient magnitude.

[0023] Marking pixel points whose target gradient amplitude is greater than a preset threshold range or within a preset threshold range as edge points, connecting the edge points, and generating a plurality of edge lines corresponding to the target image;

[0024] After segmenting the target image according to each edge line, several regional images are generated;

[0025] For each regional image, extracting a first sample regional feature, a first sample texture feature, a first sample color feature, and a first sample shape feature in the regional image, and then taking an edge line in the regional image as a first sample edge feature;

[0026] The first sample edge feature, the first sample region feature, the first sample texture feature, the first sample color feature and the first sample shape feature are used as sample ground feature corresponding to each region image;

[0027] According to the sample ground feature corresponding to each regional image, the intersections corresponding to two adjacent regional images are interpolated and filled to generate the intersection ground feature corresponding to each intersection.

[0028] Preferably, the pixel points whose target gradient amplitude is greater than a preset threshold range or is within a preset threshold range are marked as edge points, and the edge points are connected to generate a plurality of edge lines corresponding to the target image.

[0029] Marking pixel points whose target gradient amplitude is greater than a preset threshold range as strong edge points, or marking pixel points whose target gradient amplitude is within a preset threshold range as weak edge points;

[0030] After connecting each edge point according to the strong edge point and the weak edge point, several edge lines corresponding to the target image are generated.

[0031] Preferably, the interpolation filling of the intersections corresponding to two adjacent regional images is performed according to the sample ground feature corresponding to each regional image to generate the intersection ground feature corresponding to each intersection, including:

[0032] Generate the interpolation vectors corresponding to each junction according to the following formula:

[0033]

[0034] Among them, F(x,y) represents the interpolation vector corresponding to each interpolation point at the junction, w i represents the position weight of the interpolation point and the adjacent area image, x i Represents the vector corresponding to the sample feature of the interpolation point, x i is the first sample edge feature, the first sample region feature, the first sample texture feature, the first sample color feature or the first sample shape feature in the sample feature; q represents the point to be interpolated, i represents the region image adjacent to the point to be interpolated, and distance1 represents the Euclidean distance between the point to be interpolated and the adjacent region image;

[0035] Interpolation and filling are performed on the junctions corresponding to the two adjacent regional images according to each vector to be interpolated, so as to generate the junction features corresponding to each junction.

[0036] Preferably, the step of subtracting the target feature from each boundary feature to generate a difference value corresponding to each boundary feature includes:

[0037] For each boundary feature, the difference corresponding to each boundary feature is calculated according to the following formula:

[0038] distance2=cos(x a ,x b );

[0039] Where distance2 represents the difference value corresponding to the boundary feature, cos represents the cosine distance between the target feature and each boundary feature, and x a 、x b They represent the normalized boundary features and target features respectively.

[0040] Preferably, the extracting target area features, target texture features, target color features and target shape features in the aerial image includes:

[0041] Inputting the aerial image into a preset feature recognition model so that the feature recognition model extracts target area features, target texture features, target color features, and target shape features in the aerial image;

[0042] The training process of the feature recognition model includes:

[0043] Obtain sample aerial images with regional feature annotations, texture feature annotations, color feature annotations, and shape feature annotations, perform data enhancement on the sample aerial images, and obtain a number of training samples;

[0044] Iteratively train the feature recognition model according to the training samples until the feature recognition model converges;

[0045] In each training, the training sample is input into the feature recognition model to be trained, so that the feature recognition model to be trained outputs the feature prediction result corresponding to the training sample;

[0046] Compare the feature prediction results corresponding to the training samples with the actual annotation results of the training samples, and adjust the network parameters of the feature recognition model according to the comparison results;

[0047] When it is detected that the feature recognition model has converged, the feature recognition model after training is used as the feature recognition model.

[0048] Based on the above method embodiments, the present invention provides corresponding device embodiments.

[0049] An embodiment of the present invention provides a ground object recognition device based on aerial images, comprising: an image acquisition module, a ground object feature extraction module, a regional image screening module and a ground object feature matching module;

[0050] The image acquisition module is used to acquire an aerial image of a target city; wherein the target city corresponds to a plurality of regional images; each regional image corresponds to a local feature; and the boundary corresponding to two adjacent regional images corresponds to a boundary feature;

[0051] The ground feature extraction module is used to extract ground feature in the aerial image to generate target ground feature;

[0052] The regional image screening module is used to perform a subtraction between the target land feature and each boundary land feature to generate a difference value corresponding to each boundary land feature, and to use the boundary land feature corresponding to the difference value not greater than a preset difference threshold as the target boundary land feature, and then use the two adjacent regional images corresponding to the target boundary land feature as the first target regional image and the second target regional image;

[0053] The ground feature matching module is used to compare the target ground feature with the sample ground feature corresponding to the first target area image and the second target area image respectively;

[0054] When it is determined that the target object feature matches the sample object feature corresponding to the first target area successfully, a recognition result is generated for indicating that the aerial image belongs to the first target area image, or,

[0055] When it is determined that the target object feature matches the sample object feature corresponding to the second target area successfully, a recognition result is generated to indicate that the aerial image belongs to the second target area image.

[0056] Based on the above method embodiments, the present invention provides corresponding terminal device embodiments.

[0057] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, a method for identifying objects based on aerial images as described in the above-mentioned embodiment of the invention is implemented.

[0058] Based on the above method embodiments, the present invention provides corresponding storage medium item embodiments.

[0059] Another embodiment of the present invention provides a storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a method for identifying objects based on aerial images as described in the above-mentioned embodiment of the invention.

[0060] The following beneficial effects are achieved by implementing the present invention:

[0061] The embodiment of the present invention provides a method, apparatus, terminal device and storage medium for identifying objects based on aerial images. After acquiring aerial image data of a target city, the present invention first extracts the object features in the aerial image data to obtain corresponding target object features. Then, the extracted target object features are compared (subtracted) with each boundary object feature to obtain the difference between each boundary object feature and the target object feature. Based on a preset difference threshold, the boundary object features whose difference is not greater than the threshold are regarded as target boundary object features. The present invention can first compare the object features of the current aerial image with the features at the boundary of different regions. When the difference is not greater than the threshold, the matching range can be further narrowed to determine which region of the two regions the aerial image belongs to. Then, when the object features of the current aerial image are matched with the object features corresponding to the two regions, a recognition result is generated, which can determine whether the aerial image data belongs to the first target region or the second target region. Compared with the prior art, the present invention proposes a principle of intermediate image priority matching, which first performs a difference comparison between the features and the features at the junctions of each adjacent area to preliminarily screen out the junctions that are similar to the features of the target object, and then quickly locates the two target areas corresponding to the junctions for feature matching, thereby quickly screening out areas that meet the requirements based on an effective screening mechanism (such as a feature comparison mechanism at the junctions), without wasting a large amount of computing resources on areas that obviously do not meet the requirements, thereby improving the processing speed of object recognition; even when the number of areas is large, the present invention can quickly screen and locate the corresponding target areas for feature comparison and generate recognition results, thereby improving the efficiency of object recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 The present invention is a flowchart of a method for identifying ground objects based on aerial images provided by an embodiment of the present invention.

[0063] Figure 2 It is a schematic diagram of the entire process of object recognition provided by another embodiment of the present invention.

[0064] Figure 3 is a schematic diagram of a ground feature image database provided by an embodiment of the present invention;

[0065] Figure 4 It is a schematic diagram of an algorithm network architecture provided by an embodiment of the present invention.

[0066] Figure 5 The present invention is a schematic structural diagram of a ground object recognition device based on aerial images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] like Figure 1 FIG. 1 is a flow chart of a method for identifying an object based on aerial images provided by an embodiment of the present invention. The method for identifying an object based on aerial images comprises:

[0069] Step S1: Acquire an aerial image of a target city; wherein the target city corresponds to a plurality of regional images; each regional image corresponds to a local feature; and the boundary corresponding to two adjacent regional images corresponds to a boundary feature;

[0070] Step S2: extracting the ground object features in the aerial image to generate target ground object features;

[0071] Step S3: subtract the target feature from each boundary feature to generate a difference value corresponding to each boundary feature, and use the boundary feature corresponding to the difference value not greater than a preset difference threshold as the target boundary feature, and then use the two adjacent area images corresponding to the target boundary feature as the first target area image and the second target area image;

[0072] Step S4: comparing the target object features with the sample object features corresponding to the first target area image and the second target area image respectively;

[0073] When it is determined that the target object feature matches the sample object feature corresponding to the first target area successfully, a recognition result is generated for indicating that the aerial image belongs to the first target area image, or,

[0074] When it is determined that the target object feature matches the sample object feature corresponding to the second target area successfully, a recognition result is generated to indicate that the aerial image belongs to the second target area image.

[0075] For step S1, in a preferred embodiment, the present invention can identify which area of ​​the city the feature in the current aerial image belongs to, and a city can correspond to multiple regional images; each regional image will correspond to a local feature; and the intersection corresponding to two adjacent regional images can correspond to a boundary feature;

[0076] Illustratively, the present invention can pre-establish an aerial image database; the aerial image database includes a number of sample land feature and a number of boundary land feature; and each of the sample land feature corresponds to each regional image in the target city.

[0077] In a preferred embodiment, each sample feature includes: a first sample edge feature, a first sample region feature, a first sample texture feature, a first sample color feature, and a first sample shape feature;

[0078] Each boundary feature includes: a second sample edge feature, a second sample area feature, a second sample texture feature, a second sample color feature and a second sample shape feature; it can be understood that the first sample edge feature is different from the second sample edge feature.

[0079] The present invention can realize the process of automatically constructing an aerial image database, specifically:

[0080] Acquire an original aerial image corresponding to a target city; wherein the original aerial image contains images of all regions of the target city;

[0081] Perform preprocessing on each original image to generate a processed target image;

[0082] Calculate the gradient magnitude and gradient direction of each pixel in the target image. For each pixel, take the maximum gradient magnitude in the same gradient direction as the target gradient magnitude.

[0083] Marking pixel points whose target gradient amplitude is greater than a preset threshold range or within a preset threshold range as edge points, connecting the edge points, and generating a plurality of edge lines corresponding to the target image;

[0084] After segmenting the target image according to each edge line, several regional images are generated;

[0085] For each regional image, extracting a first sample regional feature, a first sample texture feature, a first sample color feature, and a first sample shape feature in the regional image, and then taking an edge line in the regional image as a first sample edge feature;

[0086] The first sample edge feature, the first sample region feature, the first sample texture feature, the first sample color feature and the first sample shape feature are used as sample ground feature corresponding to each region image;

[0087] According to the sample ground feature corresponding to each regional image, the intersections corresponding to two adjacent regional images are interpolated and filled to generate the intersection ground feature corresponding to each intersection.

[0088] Indicative, such as Figure 2 as well as Figure 3 As shown, the present invention can collect image information of objects in different areas of the city by deploying aerial photography devices in different areas according to the geographical layout of the city and the distribution characteristics of objects, thereby establishing an aerial photography image database, so that the real-time recognized features can be matched with the database later to obtain the object recognition results.

[0089] By establishing a cloud data storage system, the original image information (i.e., original aerial images) collected by each aerial photography device is received and stored, and the image clarity is enhanced through the Retinet algorithm. Then, the original image information is pre-processed, such as first converting the size of the stored data, that is, cropping all image data to the same size, and unifying the resolution and color space. In addition, all data are normalized to ensure the consistency of the data in terms of numerical values, so as to obtain the processed target image;

[0090] In order to further extract the edge and segment the region of the target image, the present invention first calculates the gradient amplitude and gradient direction of each pixel in the target image, and for each pixel, the maximum gradient amplitude in the same gradient direction is used as the target gradient amplitude; thereby, the pixel points whose target gradient amplitude is greater than a preset threshold range can be marked as strong edge points, or the pixel points whose target gradient amplitude is within the preset threshold range can be marked as weak edge points;

[0091] After connecting each edge point according to the strong edge point and the weak edge point, several edge lines corresponding to the target image are generated. Schematically, the present invention can use an edge detection algorithm (such as the Canny operator) to perform edge detection on the image. After obtaining the edge binary image, the algorithm can usually further classify the edge points, such as strong edge points and weak edge points. Usually, it is judged based on the gradient amplitude of the edge point. Points above a certain high threshold can be considered as strong edge points, while points with gradient amplitudes between the high threshold and the low threshold can be considered as weak edge points. Starting from a certain strong edge point, adjacent edge points are searched according to a certain strategy (such as neighborhood search) and connected. If the gradient amplitude of a certain adjacent point is lower than the high threshold but higher than the low threshold (i.e., a weak edge point), it can be decided whether to connect it to the line according to certain rules (such as whether the point is located in a neighborhood of the connected line), and finally several edge lines corresponding to the target image can be obtained.

[0092] Furthermore, after the target image is segmented according to each edge line, several regional images can be generated; specifically, the GrabCut algorithm can be used to initialize the image, perform GMM learning, minimize energy, and iteratively update the steps to achieve the segmentation of different ground object areas in the image.

[0093] Schematically, the edge strength and direction of each pixel can be calculated, and then a label is assigned to each pixel to classify the pixel as foreground or background, thereby realizing the extraction of edges and regions of different objects in the image.

[0094] For each regional image, the first sample edge feature, the first sample regional feature, the first sample texture feature, the first sample color feature and the first sample shape feature can be used as the sample ground feature corresponding to each regional image; and the sample ground feature feature can represent the category of the ground feature, such as buildings, roads, rivers, vegetation, etc., so that when comparing the ground feature between images later, the ground feature category of the real-time image can be determined, and it can be judged whether it belongs to the image of the region. Then the present invention can define the ground feature category of each image data (i.e., aerial image) as the ground feature identification feature value of the image data, and the feature value can be used as an index or tag for rapid retrieval and processing of ground feature data of a specific category, thereby significantly improving the efficiency of recognition and analysis of aerial images.

[0095] Furthermore, the present invention can also generate corresponding boundary feature at the boundary of two adjacent areas through interpolation filling. For example, when one aerial photography device collects image data of building category and another aerial photography device collects image data of road category, interpolation filling is performed on the nodes in the middle area of ​​the two aerial photography devices to improve the feature generation and expression of the boundary.

[0096] In a preferred embodiment, the interpolation filling of the intersections corresponding to two adjacent regional images is performed according to the sample ground feature corresponding to each regional image to generate the intersection ground feature corresponding to each intersection, including:

[0097] Generate the interpolation vectors corresponding to each junction according to the following formula:

[0098]

[0099] Among them, F(x,y) represents the interpolation vector corresponding to each interpolation point at the junction, w i represents the position weight of the interpolation point and the adjacent area image, x i Represents the vector corresponding to the sample feature of the interpolation point, x i is the first sample edge feature, the first sample region feature, the first sample texture feature, the first sample color feature or the first sample shape feature in the sample feature; q represents the point to be interpolated, i represents the region image adjacent to the point to be interpolated, and distance1 represents the Euclidean distance between the point to be interpolated and the adjacent region image;

[0100] Interpolation and filling are performed on the junctions corresponding to the two adjacent regional images according to each vector to be interpolated, so as to generate the junction features corresponding to each junction.

[0101] Then, when the real-time image is subsequently compared, it can be compared with the boundary feature in the database to achieve preliminary rapid screening. Schematically, when generating boundary feature corresponding to each boundary, a series of pixels or data points representing boundary feature are actually generated, and the features are second sample edge feature, second sample area feature, second sample texture feature, second sample color feature and second sample shape feature.

[0102] The generated boundary feature is stored in a database for subsequent comparison with the real-time image. When the real-time image arrives, its boundary feature is first extracted. Then, these features are compared with the boundary feature in the database. By comparing the similarity of the real-time image with the features in the database, possible matching images or areas can be quickly screened out, which helps to narrow the search scope and improve the efficiency of subsequent detailed comparison.

[0103] For step S2, in a preferred embodiment, the target object features include: target edge features, target area features, target texture features, target color features and target shape features;

[0104] Extracting the ground object features in the aerial image to generate target ground object features specifically includes:

[0105] Extracting target area features, target texture features, target color features, and target shape features in the aerial image, and using edge lines in the aerial image as target edge features corresponding to the aerial image;

[0106] The target edge features, target area features, target texture features, target color features and target shape features are used as target ground object features corresponding to the aerial image.

[0107] Illustratively, the present invention can extract target area features, target texture features, target color features and target shape features in aerial images through a multi-model Seq2Seq object recognition network architecture.

[0108] Then, the aerial image is input into a preset feature recognition model (i.e., a multi-model Seq2Seq object recognition network architecture) so that the feature recognition model extracts target area features, target texture features, target color features, and target shape features in the aerial image;

[0109] The training process of the feature recognition model includes:

[0110] Obtain sample aerial images with regional feature annotations, texture feature annotations, color feature annotations, and shape feature annotations, perform data enhancement on the sample aerial images, and obtain a number of training samples;

[0111] Iteratively train the feature recognition model according to the training samples until the feature recognition model converges;

[0112] In each training, the training sample is input into the feature recognition model to be trained, so that the feature recognition model to be trained outputs the feature prediction result corresponding to the training sample;

[0113] Compare the feature prediction results corresponding to the training samples with the actual annotation results of the training samples, and adjust the network parameters of the feature recognition model according to the comparison results;

[0114] When it is detected that the feature recognition model has converged, the feature recognition model after training is used as the feature recognition model.

[0115] Specifically, in each iteration, a training sample is input into the feature recognition model to be trained, and the feature recognition model outputs the feature prediction results corresponding to the training sample according to the current network parameters. These prediction results should include the prediction values ​​of regional features, texture features, color features and shape features.

[0116] The feature prediction results output by the model are compared with the actual annotation results of the training samples, and the value of the loss function (such as cross entropy loss, mean square error, etc.) is calculated. Then, based on the value of the loss function, the gradient is calculated using the back propagation algorithm, and the network parameters are updated using an optimization algorithm (such as stochastic gradient descent, Adam, etc.) to reduce the value of the loss function.

[0117] Repeat the above steps until the training termination conditions are met (such as reaching the preset number of iterations, the loss function value is less than a certain threshold, etc.), and use the feature recognition model at this time as the final model for subsequent use in target feature extraction of aerial images.

[0118] In practical applications, the aerial images to be processed are input into the trained feature recognition model to extract the target area features, target texture features, target color features and target shape features.

[0119] For step S3, in a preferred embodiment, the step of subtracting the target feature from each boundary feature to generate a difference value corresponding to each boundary feature includes:

[0120] For each boundary feature, the difference corresponding to each boundary feature is calculated according to the following formula:

[0121] distance2=cos(x a ,x b );

[0122] Where distance2 represents the difference value corresponding to the boundary feature, cos represents the cosine distance between the target feature and each boundary feature, and x a 、x b They represent the normalized boundary feature and target feature respectively. Schematically, if distance2 is greater than a preset threshold, it means that the two features do not match.

[0123] It can be understood that the present invention proposes a principle of intermediate image priority matching, which first preliminarily screens out the intersections with similar characteristics to the target object through difference comparison of the intersection features of each adjacent area, and then quickly locates the two target areas corresponding to the intersection for subsequent more accurate feature matching.

[0124] Specifically, Figure 4 As shown, the target object features (i.e., feature vectors) predicted by the multi-model Seq2Seq object recognition network architecture are first matched with the intermediate interpolated points at each junction. When the threshold formula is met, it is determined that the image belongs to a certain type of object in the pairwise adjacent regional images corresponding to the junction, and then detailed matching is performed with the regional features corresponding to the two pairwise adjacent regional images to check which type of object is collected by which aerial photography device, that is, to check which region the object image corresponds to. Finally, the source of the aerial image is determined according to the aerial photography device and the object category.

[0125] That is, the multi-model Seq2Seq object recognition network architecture can be used to extract features from the input aerial images to obtain feature vectors of the target objects (including regional features, texture features, color features, and shape features). These feature vectors are compared with the feature vectors of the pre-stored intermediate interpolation points (or feature points) at the junctions of each adjacent region. A suitable threshold is set. When the difference is less than the threshold, it is considered that the target object features are similar to the features of the junction, and these junctions can be preliminarily screened out.

[0126] In a preferred embodiment, when the intermediate interpolation data cannot match the predicted vector, that is, it cannot satisfy the threshold formula, the vector value is strongly matched with all the ground object category vectors of the aerial image database of each location to check whether there is an image database vector that satisfies the threshold formula. When none of them exists, the aerial image is determined to be an unrecognizable image.

[0127] Schematically, the image data collected in real time is recognized through the Seq2Seq algorithm model network construction. The feature item of the network input is the image data collected by the aerial photography device at 10 consecutive moments, and its output item is also a vector value. The vector is used to match the established aerial image database to find the vector that is most similar to it; schematically, the loss value is the cosine distance between the predicted vector and the closest vector. In this application, the loss value can be given by the following image loss function.

[0128] Among them, the image loss function is expressed as:

[0129]

[0130] Among them, L is the image loss value, τ is the adjustment factor, x i is the feature vector value of the image data collected by the aerial photography device in real time, that is, a vector composed of five types of data: texture, color, shape, edge, and area. There are many types, that is, different types of objects. n indicates the total amount of data contained in the aerial image database collected by the aerial photography device at this location, and y i It is the i-th data vector collected by the aerial photography device.

[0131] For step S4, in a preferred embodiment, the present invention can obtain sample ground object features corresponding to the first target area image and the second target area image from a pre-stored database, and these sample ground object features should represent the ground object characteristics unique to each area.

[0132] The target feature vector extracted in step S2 is compared with the sample feature vector corresponding to the first target region image. At the same time, the target feature vector is also compared with the sample feature vector corresponding to the second target region image. Schematically, the comparison method can be by calculating the distance between feature vectors (such as Euclidean distance, cosine similarity, etc.) or other similarity measurement indicators.

[0133] The process of determining whether a match is made is as follows:

[0134] A suitable threshold is set to determine whether the target feature matches the sample feature. If the distance or similarity between the target feature and the sample feature corresponding to the first target area satisfies the threshold, the match is determined to be successful.

[0135] Similarly, if the target object feature matches the sample object feature corresponding to the second target area successfully, a corresponding determination result will also be obtained.

[0136] Then, when it is determined that the target object feature matches the sample object feature corresponding to the first target area successfully, a recognition result for indicating that the aerial image belongs to the first target area image can be generated.

[0137] If the sample ground feature corresponding to the second target area is matched successfully, a recognition result is generated for indicating that the aerial image belongs to the second target area image.

[0138] Finally, the generated recognition results can be output for subsequent processing or display. Through the above-mentioned feature comparison and matching methods, the target area to which the aerial image belongs can be accurately determined, providing a reliable basis for subsequent image analysis, target recognition and other tasks.

[0139] The present invention can quickly screen out areas that meet the requirements based on an effective screening mechanism (such as a feature comparison mechanism at the junction), without wasting a large amount of computing resources on areas that obviously do not meet the requirements, thereby improving the processing speed of object recognition; even when the number of areas is large, the present invention can quickly screen and locate the corresponding target area for feature comparison and generate recognition results, thereby improving the efficiency of object recognition.

[0140] like Figure 5 As shown, based on the above-mentioned various embodiments of the method for identifying ground objects based on aerial images, the present invention provides a corresponding device embodiment;

[0141] An embodiment of the present invention provides a ground object recognition device based on aerial images, comprising: an image acquisition module, a ground object feature extraction module, a regional image screening module and a ground object feature matching module;

[0142] The image acquisition module is used to acquire an aerial image of a target city; wherein the target city corresponds to a plurality of regional images; each regional image corresponds to a local feature; and the boundary corresponding to two adjacent regional images corresponds to a boundary feature;

[0143] The ground feature extraction module is used to extract ground feature in the aerial image to generate target ground feature;

[0144] The regional image screening module is used to perform a subtraction between the target land feature and each boundary land feature to generate a difference value corresponding to each boundary land feature, and to use the boundary land feature corresponding to the difference value not greater than a preset difference threshold as the target boundary land feature, and then use the two adjacent regional images corresponding to the target boundary land feature as the first target regional image and the second target regional image;

[0145] The ground feature matching module is used to compare the target ground feature with the sample ground feature corresponding to the first target area image and the second target area image respectively;

[0146] When it is determined that the target object feature matches the sample object feature corresponding to the first target area successfully, a recognition result is generated for indicating that the aerial image belongs to the first target area image, or,

[0147] When it is determined that the target object feature matches the sample object feature corresponding to the second target area successfully, a recognition result is generated to indicate that the aerial image belongs to the second target area image.

[0148] It should be noted that the device embodiments described above are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, and may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement it without paying creative labor.

[0149] Those skilled in the art can clearly understand that, for the sake of convenience and simplicity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0150] Based on the above-mentioned various embodiments of the method for identifying objects based on aerial images, the present invention provides corresponding embodiments of terminal equipment items.

[0151] An embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a method for identifying objects based on aerial images as described in any method embodiment of the present invention is implemented.

[0152] The terminal device may be a computing terminal device such as a desktop computer, a notebook, a palm computer, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0153] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0154] The memory can be used to store the computer program, and the processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device or other volatile solid-state storage device.

[0155] Based on the above-mentioned various embodiments of the method for identifying ground objects based on aerial images, the present invention provides corresponding storage medium embodiments.

[0156] An embodiment of the present invention provides a storage medium, wherein the storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a method for identifying objects based on aerial images as described in any method embodiment of the present invention.

[0157] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each method embodiment described above can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0158] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for identifying ground objects based on aerial images, characterized in that: include: Acquire an aerial image of a target city; wherein the target city corresponds to a plurality of regional images; each regional image corresponds to a local feature; and a junction corresponding to two adjacent regional images corresponds to a junction feature; Extracting the features of the ground objects in the aerial image to generate the features of the target ground objects; Subtract the target feature from each boundary feature to generate a difference value corresponding to each boundary feature, and use the boundary feature corresponding to the difference value not greater than a preset difference threshold as the target boundary feature, and then use the adjacent area images corresponding to the target boundary feature as the first target area image and the second target area image; Comparing the target ground feature with the sample ground feature corresponding to the first target area image and the second target area image respectively; When it is determined that the target object feature matches the sample object feature corresponding to the first target area successfully, a recognition result is generated for indicating that the aerial image belongs to the first target area image, or, When it is determined that the target object feature matches the sample object feature corresponding to the second target area successfully, a recognition result is generated to indicate that the aerial image belongs to the second target area image.

2. A method for identifying ground objects based on aerial images as claimed in claim 1, characterized in that: The target object features include: target edge features, target area features, target texture features, target color features and target shape features; The step of extracting the ground object features in the aerial image to generate target ground object features includes: Extracting target area features, target texture features, target color features, and target shape features in the aerial image, and using edge lines in the aerial image as target edge features corresponding to the aerial image; The target edge features, target area features, target texture features, target color features and target shape features are used as target ground object features corresponding to the aerial image.

3. The method for identifying ground objects based on aerial images according to claim 1, characterized in that: Also includes: Aerial image database; the aerial image database includes a number of sample ground features and a number of boundary ground features; Each local feature corresponds to each regional image in the target city; Each sample feature includes: a first sample edge feature, a first sample region feature, a first sample texture feature, a first sample color feature, and a first sample shape feature; each boundary feature includes: a second sample edge feature, a second sample region feature, a second sample texture feature, a second sample color feature, and a second sample shape feature; The construction of the aerial image database includes: Acquire an original aerial image corresponding to a target city; wherein the original aerial image contains images of all regions of the target city; Perform preprocessing on each original image to generate a processed target image; Calculate the gradient magnitude and gradient direction of each pixel in the target image. For each pixel, take the maximum gradient magnitude in the same gradient direction as the target gradient magnitude. Marking pixel points whose target gradient amplitude is greater than a preset threshold range or within a preset threshold range as edge points, connecting the edge points, and generating a plurality of edge lines corresponding to the target image; After segmenting the target image according to each edge line, several regional images are generated; For each regional image, extracting a first sample regional feature, a first sample texture feature, a first sample color feature, and a first sample shape feature in the regional image, and then taking an edge line in the regional image as a first sample edge feature; The first sample edge feature, the first sample region feature, the first sample texture feature, the first sample color feature and the first sample shape feature are used as sample ground feature corresponding to each region image; According to the sample ground feature corresponding to each regional image, the intersections corresponding to two adjacent regional images are interpolated and filled to generate the intersection ground feature corresponding to each intersection.

4. The method for identifying ground objects based on aerial images according to claim 3, characterized in that: The pixel points whose target gradient amplitude is greater than or within a preset threshold range are marked as edge points, and the edge points are connected to generate a plurality of edge lines corresponding to the target image. Marking pixel points whose target gradient amplitude is greater than a preset threshold range as strong edge points, or marking pixel points whose target gradient amplitude is within a preset threshold range as weak edge points; After connecting each edge point according to the strong edge point and the weak edge point, several edge lines corresponding to the target image are generated.

5. The method for identifying ground objects based on aerial images according to claim 4, characterized in that: The interpolation filling is performed on the intersections corresponding to two adjacent regional images according to the sample ground feature corresponding to each regional image to generate the intersection ground feature corresponding to each intersection, including: Generate the interpolation vectors corresponding to each junction according to the following formula: Among them, F(x,y) represents the interpolation vector corresponding to each interpolation point at the junction, w i represents the position weight of the interpolation point and the adjacent area image, x i Represents the vector corresponding to the sample feature of the interpolation point, x i is the first sample edge feature, the first sample region feature, the first sample texture feature, the first sample color feature or the first sample shape feature in the sample feature; q represents the point to be interpolated, i represents the region image adjacent to the point to be interpolated, and distance1 represents the Euclidean distance between the point to be interpolated and the adjacent region image; Interpolation and filling are performed on the junctions corresponding to the two adjacent regional images according to each vector to be interpolated, so as to generate the junction features corresponding to each junction.

6. A method for identifying ground objects based on aerial images as claimed in claim 5, characterized in that: Subtracting the target feature from each boundary feature to generate a difference value corresponding to each boundary feature includes: For each boundary feature, the difference corresponding to each boundary feature is calculated according to the following formula: distance2=cos(x a ,x b ); Where distance2 represents the difference value corresponding to the boundary feature, cos represents the cosine distance between the target feature and each boundary feature, and x a 、x b They represent the normalized boundary features and target features respectively.

7. The method for identifying ground objects based on aerial images according to claim 2, characterized in that: The extracting of target area features, target texture features, target color features and target shape features in the aerial image includes: Inputting the aerial image into a preset feature recognition model so that the feature recognition model extracts target area features, target texture features, target color features, and target shape features in the aerial image; The training process of the feature recognition model includes: Obtain sample aerial images with regional feature annotations, texture feature annotations, color feature annotations, and shape feature annotations, perform data enhancement on the sample aerial images, and obtain a number of training samples; Iteratively train the feature recognition model according to the training samples until the feature recognition model converges; In each training, the training sample is input into the feature recognition model to be trained, so that the feature recognition model to be trained outputs the feature prediction result corresponding to the training sample; Compare the feature prediction results corresponding to the training samples with the actual annotation results of the training samples, and adjust the network parameters of the feature recognition model according to the comparison results; When it is detected that the feature recognition model has converged, the feature recognition model after training is used as the feature recognition model.

8. A ground object recognition device based on aerial images, characterized in that: include: Image acquisition module, ground feature extraction module, regional image screening module and ground feature matching module; The image acquisition module is used to acquire an aerial image of a target city; wherein the target city corresponds to a plurality of regional images; each regional image corresponds to a local feature; and the boundary corresponding to two adjacent regional images corresponds to a boundary feature; The ground feature extraction module is used to extract ground feature in the aerial image to generate target ground feature; The regional image screening module is used to perform a subtraction between the target land feature and each boundary land feature to generate a difference value corresponding to each boundary land feature, and to use the boundary land feature corresponding to the difference value not greater than a preset difference threshold as the target boundary land feature, and then use the two adjacent regional images corresponding to the target boundary land feature as the first target regional image and the second target regional image; The ground feature matching module is used to compare the target ground feature with the sample ground feature corresponding to the first target area image and the second target area image respectively; When it is determined that the target object feature matches the sample object feature corresponding to the first target area successfully, a recognition result is generated for indicating that the aerial image belongs to the first target area image, or, When it is determined that the target object feature matches the sample object feature corresponding to the second target area successfully, a recognition result is generated to indicate that the aerial image belongs to the second target area image.

9. A terminal device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a method for identifying objects based on aerial images as described in any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is executed, the device where the storage medium is located is controlled to execute a method for identifying objects based on aerial images as described in any one of claims 1 to 7.

Images