Non-agricultural remote sensing monitoring knowledge base efficient retrieval and auxiliary identification method and system

Abstract

The application belongs to the field of remote sensing image natural resource monitoring, and specifically discloses a non-agricultural remote sensing monitoring knowledge base efficient retrieval and auxiliary identification method and system, which comprises the following steps: processing of historical image and corresponding field photo data of natural resource ground surface elements and construction of a knowledge base; definition of deep feature descriptors and large-scale image feature description extraction; construction of a feature library hierarchical index based on feature similarity clustering; extraction of image feature descriptors to be retrieved; multi-level similarity measurement and N-neighbor interactive real-time retrieval; and real-time knowledge base information annotation to complete auxiliary identification of cultivated land non-agricultural difficult image. The application automatically constructs a knowledge base containing a large amount of natural resource images based on historical survey data, and constructs an efficient and accurate feature descriptor and hierarchical index structure, so as to help the workers to realize interactive real-time image retrieval and reliable difficult image identification, thereby significantly improving the effectiveness of cultivated land non-agricultural monitoring field verification work and realizing cost reduction and efficiency increase of wide-range ground surface natural resource periodic monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image natural resource monitoring, and particularly relates to a non-agricultural remote sensing monitoring knowledge base efficient retrieval and auxiliary identification method and system. BACKGROUND

[0002] Natural resource ground element information investigation, statistics and all-weather, full-coverage remote sensing intelligent monitoring are the ultimate goal of national land information investigation. The existing ground element intelligent extraction technology based on deep learning still faces the problem that the automatic interpretation result of complex remote sensing scene in a wide range is unreliable, and cannot be directly applied to the automatic monitoring task of ground element. At present, the extraction and monitoring of cultivated land resource information still rely on a combination of remote sensing image manual discrimination and field investigation. The in-house staff interprets the remote sensing image visually to monitor the cultivated land resource elements and label the non-agricultural range of suspected cultivated land, and the field staff checks the in-house labeled results on site, and takes photos as evidence and feedback. Due to the limited resolution of remote sensing images, large environmental differences and other characteristics, it is difficult for in-house staff to accurately distinguish the non-agricultural use of suspected cultivated land in remote sensing images, especially small ground objects such as temporary buildings, high-voltage towers, wind power facilities, and the like, which are similar in appearance to agricultural greenhouses, as well as landfill and ploughed cultivated land, ponds and paddy fields, and wind turbines, oil wells, and communication base stations. In order to improve the recall rate of suspected cultivated land non-agricultural monitoring, a large number of images that cannot be accurately identified are often labeled as suspected image patches, which not only increases the workload of field checking, but also significantly increases the cost, especially for large-scale, continuous and periodic monitoring tasks. It is extremely important to improve the reliability of in-house image identification results.

[0003] In the prior art, there is also a method for improving the retrieval speed by establishing an efficient index structure for the feature library, but the feature structure is not optimized, and the contradiction between the feature description ability and the length is not resolved. When constructing a massive image feature library for remote sensing monitoring applications, the feature library file is large, and it is difficult to compensate for the I / O bottleneck caused by large file reading through an efficient index structure; the neural network model used for feature extraction is trained relying on sample labeling information, such as (applied to remote sensing image typical scene recognition) but the construction cost of the sample dataset containing labeling information is high; and the feature retrieval library composed of limited sample images cannot meet the efficient retrieval demand of the difficult image knowledge base in the application of cultivated land resource monitoring in complex scenes in a wide range. The final result obtained by the existing similar image retrieval is a most similar image or a set of similar images, and the similar images retrieved may not accurately match the user's demand, lack of user participation, and the effectiveness of the retrieval result is low; the retrieval technology is only used to obtain similar images, and lacks effective application of the retrieval result. SUMMARY

[0004] To solve the problems in the prior art, the application provides a non-agricultural remote sensing monitoring knowledge base efficient retrieval and auxiliary identification method and system, which fully utilizes the massive data of typical natural resource surface element images accumulated in previous years, corresponding field investigation verified photos and attributes, constructs a knowledge base applied to low resolution remote sensing image auxiliary identification, retrieves the specified number of most similar images of a user requested image in real time in a feature library, determines and draws the field photos and attribute information of the matching images through subjective judgment of the user, realizes reliable identification of difficult to identify remote sensing images, and solves the problems mentioned in the background art.

[0005] To achieve the above object, the application provides the following technical scheme: a non-agricultural remote sensing monitoring knowledge base efficient retrieval and auxiliary identification method, comprising the following steps:

[0006] S10, natural resource surface element historical image and corresponding field photo data processing and knowledge base construction;

[0007] S11, defining a deep feature descriptor and constructing a feature extraction model, extracting a large-scale image feature description by using the constructed feature extraction model, and forming a sample feature library;

[0008] S12, constructing a hierarchical efficient index structure of the feature library based on feature similarity clustering;

[0009] S13, after enhancement processing of the image to be retrieved, completing feature descriptor extraction by using the feature extraction model; and realizing multi-level similarity measurement and N-neighbor interactive real-time retrieval based on the massive feature library;

[0010] S14, reordering the retrieval results based on an image histogram, and real-time drawing knowledge base data, comprehensively retrieving result images and field photos and attribute information provided by the knowledge base, and completing reliable identification of difficult images in non-agricultural monitoring of cultivated land.

[0011] Preferably, in step S10, the following steps are specifically included:

[0012] S20, collecting natural resource remote sensing images, field photos, and ground object coverage category data, specifically including: different phase remote sensing images accumulated by each directly subordinate surveying and mapping bureau in natural resource surface element monitoring tasks in previous years, element vector map patches matched with the remote sensing images, and element object corresponding field photos, element coverage category attributes and other information;

[0013] S21, ground object element object image sampling, corresponding field photo and attribute data coding and knowledge base creation;

[0014] S22, sample image preprocessing and data enhancement.

[0015] Preferably, in step S21, the following steps are specifically included:

[0016] cropping the feature range of the remote sensing image according to the ground feature element vector boundary information from the remote sensing image;

[0017] The cropped sample image and the corresponding field photo are named by a unique identification code generated in real time based on a timestamp in a sample image cropping process, and each remote sensing image sample is associated with corresponding multiple field photo data and attribute data through the identification code;

[0018] The field photo and attribute data are managed by a distributed file storage database, so that the corresponding field photo and attribute data can be quickly obtained through the image identification code, and the knowledge base is created.

[0019] Preferably, in step S22, specifically comprising:

[0020] The aspect ratio of the sample image is represented as R, and the calculation method is: R = max(W, H) / min(W, H), wherein W and H are the number of pixels occupied by the width and height of the sample image, respectively;

[0021] The maximum aspect ratio R1 is set to 1.5, and the maximum aspect ratio R2 is set to 3;

[0022] When the aspect ratio R of the sample image is greater than R2, the image is discarded, and when R2 >= R > R1, the image is center cropped to keep the aspect ratio not more than R1 while retaining as much image content as possible;

[0023] Through the image random enhancement method, multiple enhanced image copies of each image are obtained, and all enhanced images of each sample are stacked along the channel dimension to realize diversified visual feature enhancement expression of the ground feature element.

[0024] Preferably, in step S11, specifically comprising the following:

[0025] S23, depth feature descriptor definition and feature extraction model construction:

[0026] The depth feature descriptor definition refers to: designing a one-dimensional vector V = [v0, v1,...v2047] with a length of 2048 to uniformly describe each image feature, and storing all elements v0-v2047 of the feature vector through a half-precision floating-point number format. 2047 i

[0027] The feature extraction model construction refers to: selecting a lightweight feature extraction network encoder pre-trained on the ImageNet dataset, such as ResNet18, and the input image space size of the feature encoder is 64*64 pixels, and through dimension transformation, the last layer convolutional feature obtained by the feature extractor is used as the image uniform feature descriptor of the embodiment;

[0028] ​​S24, image feature description extraction and feature library construction:

[0029] In the image feature extraction, the multiple data copies obtained by the image random enhancement method are stacked, and the features are extracted in parallel, and the average value of the features of each image copy is taken as the final feature descriptor;

[0030] Different sample feature libraries and corresponding knowledge bases are constructed by organically combining the ground cover characteristics of different regions to obtain more reliable retrieval results.

[0031] Preferably, in step S12, an unsupervised clustering algorithm is used to perform similarity clustering on the image features, wherein the number of cluster centers is set to M is the total number of features, and based on the obtained average features of each cluster center, a first index structure is constructed, and on this basis, a second index structure is constructed for all features contained by each cluster center.

[0032] Preferably, in step S13, the following steps are included:

[0033] S26, extraction of image feature descriptor to be searched:

[0034] The data enhancement method is used to enhance the user-submitted image to be searched to obtain multiple transformed image copies;

[0035] Based on the constructed pre-training feature extraction model, stable feature descriptors are extracted for the processed image to be searched;

[0036] S27, multi-level feature similarity measurement and N-neighbor interactive real-time retrieval:

[0037] In the similar feature retrieval, first, the feature descriptor of the image to be searched is matched with the nearest cluster center from the first index structure, the similarity measurement algorithm is Euclidean distance, and the calculation method of the distance D of features V1 and V2 is: Wherein, i represents the index value of the image feature vector V;

[0038] Then, similarity retrieval is performed in the feature library related to the cluster center with the smallest feature distance, and the distance between the feature to be searched and the sample features contained in the cluster center is calculated one by one.

[0039] The user's search result each time is the N features in the feature library that are most similar to the image feature to be searched and the corresponding image unique identification code.

[0040] Preferably, in step S14, the following steps are included:

[0041] S28, search result visualization and matching image field photo interpretation: through the returned N image identification code, the original image of the search result is obtained automatically, and the search result image is reordered according to the image histogram, then the reordered search result image is visualized, the user judges the image most matched with the image to be searched, and the corresponding field photo and attribute information in the interactive interpretation knowledge base is matched;

[0042] S29, cultivated land non-agricultural difficult image knowledge base information comprehensive auxiliary identification: the user comprehensively searches the result image, the field photo and attribute information provided by the knowledge base, and realizes accurate identification of the cultivated land non-agricultural difficult image.

[0043] On the other hand, in order to achieve the above-mentioned purpose, the present application also provides the following technical scheme: a non-agricultural remote sensing monitoring knowledge base efficient search and auxiliary identification system, the system comprises:

[0044] The first data obtaining unit is used for obtaining natural resource remote sensing image data, element object field photo data and ground object coverage category information data.

[0045] The first data processing unit is used for ground object element object sample image cropping, image unique identification code generation and corresponding field photo and attribute data coding and knowledge base creation.

[0046] The second data obtaining unit is used for obtaining the user-provided difficult image to be searched.

[0047] The second data processing unit is used for pre-processing the user-submitted image to be searched, including long-width ratio over-limit cropping and data enhancement operation.

[0048] The data storage unit is used for storing the knowledge base for organizing and managing ground object element information in the form of geographic database, so as to facilitate efficient query and interpretation of field photo and attribute information through image identification code.

[0049] The model construction unit is used for defining a deep feature descriptor and constructing a feature extraction model.

[0050] The first feature construction unit is used for extracting features of the mass sample images obtained by the first data processing unit based on the feature extraction model constructed by the model construction unit, to form a sample feature library.

[0051] The second feature construction unit is used for extracting stable feature descriptors of the user-requested image to be searched after being processed by the second data processing unit based on the feature extraction model constructed by the model construction unit.

[0052] The index creation unit is used for creating a hierarchical index structure for the sample feature library.

[0053] Similarity search unit: used for similarity measurement according to the user-submitted image features to be searched from the constructed hierarchical index structure sample feature library, and returning the user-specified N most similar features and corresponding unique identification codes;

[0054] First result interaction unit: used for visual display of the corresponding original images reordered according to the image content of the N most similar features searched, interactive mapping of image data from the knowledge base through unique identification codes, and selection of the most matched image by the user according to the visual result;

[0055] Second result interaction unit: used for efficient mapping of corresponding field photo data and attribute information from the knowledge base according to the user-selected matched image;

[0056] Comprehensive recognition unit: used for comprehensive recognition of the cultivated land non-agricultural difficult image according to the similar search result image and the field photo and attribute information provided by the knowledge base.

[0057] The present application has the beneficial effects that: the present application utilizes the mass accumulated low-resolution remote sensing image objects of surface elements and corresponding field high-resolution photos and attribute data such as ground object coverage types to construct a knowledge base, samples large-scale historical remote sensing images and constructs a lightweight feature library and an efficient hierarchical index structure, realizes real-time search of similar images and the knowledge base, and assists remote sensing image interpreters in accurately recognizing difficult-to-identify suspected non-agricultural targets in low-resolution satellite images through field knowledge base mapping, thereby improving the reliability of remote sensing image interpretation results, reducing costs and increasing efficiency for field investigation work, and no related technology has been found in the efficient search and auxiliary recognition of the knowledge base based on natural resource historical images. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 It is a method step flowchart of the present application;

[0059] Figure 2 It is a method specific flowchart of Example 1 of the present application;

[0060] Figure 3 It is a flowchart of the method of identifying agricultural greenhouses by using the non-agricultural remote sensing monitoring knowledge base efficient search and auxiliary recognition method in Example 2 of the present application;

[0061] Figure 4 It is a module diagram of the non-agricultural remote sensing monitoring knowledge base efficient search and auxiliary recognition system in Example 3 of the present application;

[0062] In the figure, 1-first data obtaining unit; 2-first data processing unit; 3-data storage unit; 4-second data obtaining unit; 5-second data processing unit; 6-model construction unit; 7-first feature construction unit; 8-index creation unit; 9-similarity retrieval unit; 10-second feature construction unit; 11-first result interaction unit; 12-second result interaction unit; 13-comprehensive recognition unit. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0064] Embodiment 1

[0065] Please refer to Figure 1 and Figure 2 , the present application provides a technical solution: a non-agricultural remote sensing monitoring knowledge base efficient retrieval and auxiliary recognition method, as shown in Figure 1 , comprising the following steps:

[0066] S10, natural resource surface element historical image and corresponding field photo data processing and knowledge base construction;

[0067] S11, defining a deep feature descriptor and constructing a feature extraction model, using the constructed feature extraction model to extract large-scale image feature description, forming a sample feature library;

[0068] S12, construction of hierarchical efficient index structure of feature library based on feature similarity clustering;

[0069] S13, after enhancement processing of the image to be retrieved, using the feature extraction model to complete the feature descriptor extraction; and based on the massive feature library, realizing multi-level similarity measurement and N-neighbor interactive real-time retrieval;

[0070] S14, reordering the retrieval results based on the image histogram, and real-time mapping the knowledge base data, combining the retrieval result image with the field photo and attribute information provided by the knowledge base, to complete the reliable recognition of the cultivated land non-agricultural monitoring difficult image.

[0071] Further, in step S10, as shown in Figure 2 , specifically comprising the following:

[0072] S20, collecting natural resource remote sensing images, field photos, and ground object coverage category data, specifically including: collecting different time remote sensing images accumulated by national direct mapping bureaus or other related parts in previous years, element vector map patches matched with remote sensing images, and element object corresponding field photos, element coverage categories and other attribute information.

[0073] In addition, the remote sensing image generally refers to low-resolution satellite image, and the general resolution is lower than 0.5 meters. There are problems such as unclear visual features of small ground object, confusion of visual features between similar ground object categories, and difficulty in accurate identification by manual visual inspection.

[0074] The matched element vector map patch is the boundary information of various ground object obtained by manual visual interpretation of historical natural resource ground element monitoring tasks, which provides reference for sample image cropping.

[0075] The field photo is generally high-resolution evidence photo data obtained by field shooting based on internal interpretation results, which can accurately express the field situation of each ground object.

[0076] S21, ground object element object image sampling, corresponding field photo and attribute data coding and knowledge base creation; specifically including:

[0077] Specifically, the data processing includes element sample image cropping, image unique identification code generation, field photo and attribute data knowledge base construction and the like.

[0078] First, each ground object element object with field photo is cropped from large-scale remote sensing image. In order to make each sample image express only the visual features of a specific ground object category, we crop the remote sensing image in the element range from the remote sensing image according to the ground element vector boundary information.

[0079] Since the boundary of the ground object is usually irregular, the sample image is obtained by cropping through the minimum circumscribed rectangle range, and the cropped sample image and its corresponding field photo are named by the real-time generated unique identification code.

[0080] In addition, a single sampled image sample usually corresponds to multiple field photo data taken from different angles. The corresponding multiple field photos are named in order by two-digit serial numbers, for example, "identification code_01".

[0081] The identification code format is a string of 32 hexadecimal numbers, which is generated in real time based on the timestamp in the sample image cropping process. The identification code has global uniqueness, and each remote sensing image sample is associated with its corresponding multiple field photo data and attribute data through the identification code.

[0082] The field photos and attribute data are organized and managed through a distributed file storage database (MongoDB), so that the corresponding field photos and attribute data can be quickly obtained through the image identification code.

[0083] The attribute data includes administrative division information, coverage category information, and land area of the feature object.

[0084] The created knowledge base is a geographic database formed by efficient organization and management of the above data, and the corresponding field photos and land attribute information of the low-resolution remote sensing image object can be associated through the low-resolution remote sensing image object. Conversely, the category of the low-resolution image object can be accurately determined through the field photos and attribute information, and the creation of the knowledge base is completed.

[0085] S22, sample image preprocessing and data enhancement.

[0086] Specifically, due to the differences in size, resolution, aspect ratio, etc. of the sampled sample images, the embodiment limits the pixel range of the shortest side L1 of the sample image to [16, 256], which is obtained according to the estimation of the difficult-to-identify feature element scale of the actual remote sensing image, and can be adjusted appropriately in actual application.

[0087] In addition, the aspect ratio of the sample image in the embodiment is represented as R, and the calculation method is: R = max (W, H) / min (W, H), where W and H are the number of pixels occupied by the width and height of the sample image, respectively.

[0088] Since the input image of the feature extraction model is of fixed size, in order to avoid the problem that the extracted feature descriptors are not robust due to the serious visual feature deformation caused by image scaling, the embodiment sets the maximum aspect ratio R1 to 1.5 and the ultra-limit aspect ratio R2 to 3.

[0089] When the aspect ratio R of the sample image is greater than R2, it is discarded, and when R2 >= R > R1, the image is center cropped to keep the aspect ratio of the image not exceeding R1 while retaining as much image content as possible.

[0090] In order to improve the inconsistency of image color, translation, rotation and other factors on feature expression, the embodiment obtains multiple enhanced image copies of each image through image random enhancement method, and stacks all enhanced images of each sample along the channel dimension to realize diversified visual feature enhancement expression of feature objects, and provides support for improving the robustness of image features in complex environments.

[0091] The random image enhancement method includes random color enhancement, random horizontal or vertical flipping, random n*90-degree rotation, and obtaining an Image_A with a dimension of (W*H*4C) after enhancement processing of a sample image Image with an original dimension of (W*H*C) through channel dimension stacking of four image copies.

[0092] Further, in step S11, the following is specifically included:

[0093] S23, depth feature descriptor definition and feature extraction model construction:

[0094] Depth feature descriptor definition: Specifically, in order to consider accurate image feature description and efficient large-scale image feature retrieval, the embodiment designs a one-dimensional vector V = [v0, v1,...v 2047 ] with a length of 2048 to uniformly describe each image feature.

[0095] In order to reduce the memory space occupation of the feature sample library and improve the efficiency of the feature library similarity retrieval, the embodiment stores all elements v i of the feature vector in half-precision floating-point format; thereby significantly reducing the space occupation of the feature while ensuring high feature descriptor expression ability, avoiding the I / O performance bottleneck caused by the massive feature library.

[0096] Feature extraction model construction: In terms of feature extraction model design, a lightweight feature extraction network encoder pre-trained on the ImageNet dataset is selected, such as ResNet18. The input image space size of the feature encoder is 64*64 pixels. Through dimension transformation, the last layer convolutional feature obtained by the feature extractor is used as the image uniform feature descriptor of the embodiment.

[0097] S24, image feature description extraction and feature library construction:

[0098] Specifically, during image feature extraction, multiple data copies obtained by the image random enhancement method are stacked, and features are extracted in parallel. The average value of the features of each image copy is taken as the final feature descriptor.

[0099] The pre-trained lightweight feature extraction model does not need to obtain additional supervised information for training or optimization, and realizes efficient extraction of feature descriptors of enhanced sample images through an unsupervised strategy.

[0100] Due to the significant differences in ground cover types and characteristics with geographical location, climate, customs and other factors in the country, in the construction of the sample feature library, the embodiment can combine the ground cover characteristics of different provinces, cities, districts (counties) according to actual engineering needs, and construct different sample feature libraries and corresponding knowledge bases, further improving the efficiency and accuracy of subsequent similar image retrieval, to obtain more reliable retrieval results.

[0101] Further, in step S12, that is, step S25: construction of a feature library hierarchical efficient index structure based on feature similarity clustering.

[0102] Since the sample library usually contains hundreds of thousands of feature vectors, in order to realize large-scale image feature real-time retrieval, the embodiment first uses an unsupervised clustering algorithm to perform similarity clustering on the image features;

[0103] The clustering algorithm used in the embodiment is K-means, which can be replaced according to actual engineering needs, and its method includes but is not limited to K-means clustering, mean shift clustering, Gaussian mixture model clustering, and hierarchical clustering. The number of cluster centers is set to M is the total number of features, and based on the obtained average feature of each cluster center, a first-level index structure is constructed, and on this basis, a second-level index structure is constructed for all features contained in each cluster center.

[0104] Further, in step S13, as shown in Figure 2 , it specifically includes the following:

[0105] S26, extracting the image feature descriptor to be retrieved.

[0106] Specifically, to improve the efficiency of user retrieval of similar images, the embodiment supports users to submit images to be retrieved in multiple ways, including but not limited to local image loading, remote sensing image region sketching, screen capture, etc.

[0107] First, the data enhancement method described in step S22 is used to enhance the user-submitted image to be retrieved, and a plurality of transformed image copies are obtained;

[0108] Then, based on the pre-trained feature extraction model constructed in step S23, a stable feature descriptor is extracted for the processed image to be retrieved.

[0109] S27, multi-level feature similarity measurement and N-neighbor interactive real-time retrieval.

[0110] In similar feature retrieval, first, the feature descriptor of the image to be retrieved is matched with the closest cluster center from the first-level index structure. The similarity measurement algorithm used in the embodiment is Euclidean distance. Assuming that the features V1 and V2, the distance D is calculated as follows: i represents an index value of the image feature vector V;

[0111] Then, the feature library involved in the cluster center with the minimum feature distance is searched for similarity, and the distance between the feature to be searched and the sample feature included in the cluster center is calculated one by one;

[0112] In addition, the embodiment provides free setting of the number N of search results for the user, and the user searches for N features most similar to the image feature to be searched and the corresponding image unique identification code in the feature library each time, thereby providing support for interactive judgment of the user.

[0113] Each search only needs to compare the similarity of two features, thereby significantly reducing the number of times of calculation of the feature distance and improving the search efficiency.

[0114] Further, in step S14, the following is specifically included:

[0115] S28, search result visualization and matching image field photo mapping.

[0116] Since the search results are sorted according to the similarity of the image features, in order to enable the user to quickly obtain effective knowledge base information, the returned search results are reordered first.

[0117] Through the N image identification codes returned, the corresponding images are automatically obtained, and the images are reordered according to the image histogram, so that the results more similar to the content of the image to be searched are placed in the front, and the corresponding images of the reordered search results are automatically visualized;

[0118] The user subjectively judges the image most matching the image to be searched, and maps the corresponding field photos and attribute information in the knowledge base through interactive mapping;

[0119] The embodiment associates the image unique identification code with the corresponding field photos and attribute information in the knowledge base;

[0120] The interactive selection and field photo mapping process are real-time and automatic, and the user can select the matching image patch and map the field photo in real time from the N search results at any time.

[0121] S29, cultivated land non-agriculturalization difficult image knowledge base information comprehensive auxiliary identification.

[0122] The user comprehensively searches for the image, the field photo and the attribute information provided by the knowledge base, and realizes accurate identification of the cultivated land non-agriculturalization difficult image.

[0123] Specifically, the user can comprehensively search the result image, the on-site photo and the attribute information, realize more reliable identification of the image difficult to identify, significantly improve the identification accuracy of the indoor operation personnel on the low resolution remote sensing image difficult image spot, provide more effective data support for the outdoor investigation and verification personnel based on the indoor identification of the suspected illegal use image spot, and reduce the workload and cost of the outdoor investigation.

[0124] The application fully utilizes the historical accumulated ground object remote sensing image, automatically extracts the image robust feature descriptor through the pre-trained lightweight deep feature encoder, does not need the supervision of the model training and optimization of the image category information, realizes the automatic generation of the massive remote sensing image knowledge base and the efficient construction of the feature library, provides reliable data support for the similarity search of the remote sensing image difficult object in the wide range complex scene, and further improves the effectiveness of the similar image search result by combining the automatic search with the user subjective judgment.

[0125] Embodiment 2

[0126] The effectiveness of the efficient search and auxiliary identification method of the non-agricultural remote sensing monitoring knowledge base of the application is described below by taking a suspected cultivated land non-agricultural image knowledge base search and auxiliary identification as an example, as shown in the figure. Figure 3

[0127] Firstly, the knowledge base is constructed by using the arranged massive multi-temporal historical sample images, on-site photos and attribute data, as shown in step 1 in the figure;

[0128] Based on the designed feature extraction model, the unified feature descriptor is extracted from the historical sample image, and the image feature library and the efficient hierarchical index structure are constructed, as shown in step 2 in the figure;

[0129] The user takes the low resolution image difficult to identify as the request data, sends a similar image search request to the feature search library, the system returns N most similar features searched, and the corresponding sample images are drawn after reordering according to the image content, as shown in steps 3, 4 and 5 in the figure;

[0130] The user selects the most matched image from the search result through interactive subjective judgment, and automatically and real-timely draws the corresponding high resolution photo and attribute data in the knowledge base, as shown in steps 6, 7 and 8 in the figure;

[0131] Finally, the user comprehensively analyzes and judges the difficult image by multiple interactive selection and comparison of the similar image and the corresponding on-site high resolution photo and attribute, and finally confirms that the image is an agricultural greenhouse, as shown in step 9 in the figure.

[0132] ​The application automatically constructs a knowledge base containing a large number of natural resource images based on historical investigation data, and constructs an efficient and accurate feature descriptor and hierarchical index structure, to assist the work personnel to realize interactive real-time image retrieval and reliable land class image identification, which can significantly improve the effectiveness of field verification work, and realize the cost reduction and efficiency increase of periodic non-agricultural monitoring of wide-range cultivated land resources.

[0133] In order to test the efficiency of the application in practical engineering application, we selected different historical plot quantities to test the feature library construction time, feature library file size and single retrieval time. As shown in Table 1, the sample library construction and retrieval efficiency test results of the embodiment are as follows:

[0134] Table 1 Model library construction and retrieval efficiency test results

[0135]

[0136] The values in the table are the average of ten experiments, and the results show that the model construction time and space complexity increase linearly with the number of sample libraries. When the number of historical plot samples is 1 million, the feature library construction time is about 535 seconds, the feature library file size is about 4 GB, and the single retrieval time of 27 most similar results is about 400 milliseconds. It meets the real-time retrieval requirements of similar plot of provincial natural resource surface feature object knowledge base. The deep feature descriptor designed in the application reduces the space occupation by 50% while maintaining the image feature expression ability, and combined with the efficient hierarchical index structure, it realizes the real-time similar retrieval of the million image library.

[0137] Based on the real-time and reliable similarity retrieval technology of the mass image library, the application further realizes the association and automatic mapping of the knowledge base with the field photos and attributes, effectively assists the work personnel to accurately identify the difficult images, significantly improves the effectiveness of the field verification of suspected cultivated land non-agricultural targets, and realizes the cost reduction and efficiency increase of the periodic monitoring of wide-range cultivated land resources.

[0138] Embodiment 3

[0139] Based on the same inventive concept as the above method embodiment, the embodiment of the application further provides a non-agricultural remote sensing monitoring knowledge base efficient retrieval and auxiliary identification system, which can realize the functions provided by the above embodiments, as shown in Figure 4 The system comprises:

[0140] The first data obtaining unit 1 is used to obtain natural resource remote sensing image data, element object field investigation photo data, ground object coverage category information and the like.

[0141] The first data processing unit 2 is used for ground feature object sample image cropping, image unique identification code generation and corresponding field photo rule naming, and corresponding attribute data processing and knowledge base creation.

[0142] The second data obtaining unit 4 is used for obtaining a user-provided image to be searched, which can be quickly obtained through a screenshot function.

[0143] The second data processing unit 5 is used for pre-processing of a user-submitted image to be searched, including out-of-proportion cropping, data enhancement and other operations, to provide guarantee for accurate similar image search.

[0144] The data storage unit 3 is used for storing a knowledge base for organizing and managing ground feature element information in the form of a geographic database, so as to facilitate subsequent field photo and attribute information efficient query and plotting.

[0145] The model construction unit 6 is used for designing an image feature descriptor and constructing an efficient and reliable deep feature extraction model.

[0146] The first feature construction unit 7 is based on the feature extraction model constructed by the model construction unit, extracts a robust feature library from the mass sample images obtained by the first data processing unit, and is used for similar feature search.

[0147] The second feature construction unit 10 is based on the feature extraction model constructed by the model construction unit, extracts a robust feature from the user-requested image to be searched processed by the second data processing unit, and generates input data of the similar feature search unit.

[0148] The index creation unit 8 is used for creating a hierarchical index structure for the extracted large-scale sample image feature library, and providing technical support for efficient similar feature search.

[0149] The similar search unit 9 performs similarity measurement according to the image feature submitted by the user from the constructed hierarchical index sample feature library, and returns N most similar features and corresponding unique identification codes specified by the user.

[0150] The first result interaction unit 11 displays the corresponding original image of the N most similar features searched according to image content reordering, and efficiently plots the image data from the knowledge base according to the unique identification code, so that the user can select the most matched image according to the visual result.

[0151] The second result interaction unit 12: according to the matching image selected by the user, the corresponding field photo data and attribute information are efficiently extracted from the knowledge base, and the user can select the matching result and compare at any time;

[0152] The comprehensive identification unit 13: the comprehensive identification unit is used for accurately identifying the farmland non-agricultural difficult image by combining the similar search image with the field photo and attribute information provided by the knowledge base.

[0153] The application automatically constructs the knowledge base containing a large number of natural resource images based on historical investigation data, and constructs an efficient and accurate feature descriptor and hierarchical index structure, which assists the work personnel to realize interactive knowledge base efficient retrieval and reliable land image assisted identification, can significantly improve the effectiveness of the field verification work, and realizes the cost reduction and efficiency increase of the periodic monitoring of the wide range of farmland resources.

[0154] Although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features, and any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. A method for efficient retrieval and assisted identification of a non-agricultural remote sensing monitoring knowledge base, characterized in that, Includes the following steps: S10. Processing and knowledge base construction of historical images and corresponding field photos of natural resource surface elements; S11. Define deep feature descriptors and construct a feature extraction model. Use the constructed feature extraction model to extract feature descriptions from large-scale images, forming a sample feature library. Specifically, this includes the following: S23. Definition of deep feature descriptors and construction of feature extraction models: The deep feature descriptor is defined as follows: a one-dimensional vector V = [v0, v1, ... v1] with a length of 2048. 2047 Each image feature is uniformly described, and all elements v of the feature vector are expressed in half-precision floating-point format. i storage; The feature extraction model construction refers to: selecting the lightweight feature extraction network encoder ResNet18 pre-trained on the ImageNet dataset, with the input image space size of the feature encoder being 64*64 pixels, and using the last layer of convolutional features obtained by the feature extractor as the unified image feature descriptor in this embodiment through dimensionality transformation. S24. Image Feature Description Extraction and Feature Library Construction: During image feature extraction, multiple data copies obtained by the random image augmentation method are stacked and features are extracted in parallel. The average value of the features of each image copy is then used as the final feature descriptor. By organically combining the land cover characteristics of different regions, different sample feature databases and corresponding knowledge bases are constructed to obtain more reliable retrieval results. S12. Construction of a hierarchical and efficient index structure for the feature library based on feature similarity clustering: Image features are clustered based on similarity using an unsupervised clustering algorithm, where the number of cluster centers is set to... M represents the total number of features. A first-level index structure is constructed based on the average features of each cluster center. On this basis, a second-level index structure is constructed for all features contained in each cluster center. S13. After enhancing the image to be retrieved, feature descriptors are extracted using a feature extraction model; and multi-level similarity measurement and interactive real-time retrieval with N nearest neighbors are achieved based on a massive feature library. S14. Based on the image histogram, the search results are reordered, and the knowledge base data is plotted in real time. By combining the search result images with the field photos and attribute information provided by the knowledge base, the difficult images of farmland non-agriculturalization monitoring can be reliably identified.

2. The efficient retrieval and assisted identification method for non-agricultural remote sensing monitoring knowledge base according to claim 1, characterized in that: Step S10 specifically includes the following: S20. Collect remote sensing images, field photos, and land cover category data of natural resources, specifically including: remote sensing images of different time phases accumulated by the natural resource surface element monitoring tasks of each directly affiliated surveying and mapping bureau over the years, vector vector patches of elements matched with remote sensing images, and field photos of the element objects, and element cover category attribute information. S21. Sampling of ground feature objects, corresponding field photos and attribute data encoding and knowledge base creation; S22, Sample Image Preprocessing and Data Augmentation.

3. The efficient retrieval and assisted identification method for non-agricultural remote sensing monitoring knowledge base according to claim 2, characterized in that: Step S21 specifically includes: Remote sensing images that crop the feature range from remote sensing images based on the vector boundary information of ground features; The cropped sample images and their corresponding field photos are named with a unique identifier generated in real time based on the timestamp during the sample image cropping process. Each remote sensing image sample is associated with its multiple corresponding field photo data and attribute data through the identifier. Field photos and attribute data are managed through a distributed file storage database, which facilitates the quick retrieval of corresponding field photos and attribute data using image identification codes, thus enabling the creation of a knowledge base.

4. The efficient retrieval and assisted identification method for non-agricultural remote sensing monitoring knowledge base according to claim 2, characterized in that: Step S22 specifically includes: The aspect ratio of the sample image is represented by R, and it is calculated as follows: R = max(W,H) / min(W,H), where W and H are the number of pixels occupied by the width and height of the sample image, respectively. Set the maximum aspect ratio R1 to 1.5 and the maximum aspect ratio R2 to 3; When the aspect ratio of the sample image is R>R2, it is discarded. When R2>=R>R1, the image is cropped to the center to keep the aspect ratio of the image from exceeding R1 while preserving the image content as much as possible. By using a random image augmentation method, multiple augmented image copies of each image are obtained, and all augmented images of each sample are stitched and stacked along the channel dimension to achieve a diverse visual feature enhancement expression of ground features.

5. The efficient retrieval and assisted identification method for non-agricultural remote sensing monitoring knowledge base according to claim 1, characterized in that: Step S13 specifically includes the following: S26. Extraction of feature descriptors from the image to be retrieved: Data augmentation methods are used to enhance the user-submitted images to be retrieved, resulting in multiple transformed image copies; Based on the constructed pre-trained feature extraction model, stable feature descriptors are extracted for the processed image to be retrieved; S27. Multi-level feature similarity measurement and N-nearest neighbor interactive real-time retrieval: In similarity feature retrieval, the nearest cluster centers are first matched from the first-level index structure based on the feature descriptors of the image to be retrieved. The similarity measurement algorithm is Euclidean distance. Let features V1 and V2 be used, and the distance D between them is calculated as follows: Where i represents the index value of the image feature vector V; Then, a similarity search is performed on the feature library involved in the cluster center with the smallest feature distance, and the distance between the feature to be searched and the sample features contained in the cluster center is calculated one by one; Each search result for a user consists of the N features in the feature library that are most similar to the features of the image to be searched, along with the corresponding unique image identifier.

6. The efficient retrieval and assisted identification method for non-agricultural remote sensing monitoring knowledge base according to claim 1, characterized in that: Step S14 specifically includes the following: S28. Visualization of search results and matching of field photos: The original images of the search results are automatically obtained through the N returned image identification codes, and then reordered according to the image histogram. The sorted search result images are then visualized, and the user judges the image that best matches the image to be searched. The corresponding field photos and attribute information in the knowledge base are also interactively mapped. S29. Comprehensive Auxiliary Identification of Difficult Images of Farmland Non-Agricultural Use: Users can accurately identify difficult images of farmland non-agricultural use by combining the images retrieved from the search results with the on-site photos and attribute information provided by the knowledge base.

7. A system for efficient retrieval and auxiliary identification of a non-agricultural remote sensing monitoring knowledge base, based on any one of claims 1-6, characterized in that: The system includes: First data acquisition unit (1): used to acquire remote sensing image data of natural resources, on-site photograph data of feature objects, and information data on land cover categories; First data processing unit (2): used for cropping sample images of ground feature objects, generating unique image identifiers and encoding corresponding field photos and attribute data, and creating a knowledge base; Data storage unit (3): Used to store the knowledge base for organizing and managing geographic feature information in the form of a geographic database, which facilitates efficient querying and mapping of field photos and attribute information through image identification codes; Second data acquisition unit (4): used to acquire the difficult image to be retrieved provided by the user; Second data processing unit (5): used for preprocessing the image to be retrieved submitted by the user, including aspect ratio over-limit cropping and data augmentation operations; Model building unit (6): used to define deep feature descriptors and build feature extraction models; First feature construction unit (7): Based on the feature extraction model constructed by the model construction unit (6), it extracts features from the massive sample images obtained by the first data processing unit (2) to form a sample feature library; Index creation unit (8): Used to create a hierarchical index structure for the sample feature library; Similarity retrieval unit (9): Based on the features of the image to be retrieved submitted by the user, it performs similarity measurement from the constructed hierarchical index structure sample feature library and returns the N most similar features specified by the user and their corresponding unique identifiers; Second feature construction unit (10): Based on the feature extraction model constructed by the model construction unit (6), it extracts stable feature descriptors from the user-requested image after processing by the second data processing unit (5); First result interaction unit (11): used to visualize the N most similar features of the search and their corresponding original images reordered according to the image content. The image data is interactively drawn from the knowledge base through a unique identifier code. The user selects the most matching image based on the visualization results. The second result interaction unit (12) is used to efficiently retrieve the corresponding field photo data and attribute information from the knowledge base based on the matching image selected by the user. Comprehensive identification unit (13): It is used to comprehensively identify difficult images of farmland non-agriculturalization by combining similar search result images with on-site photos and attribute information provided by the knowledge base.

Images