Remote sensing image automatic interpretation system and method

Abstract

The present invention provides a system and method for automatic interpretation of objects in remote sensing images. The system includes: an automatic sample collection service, which randomly collects sample label data in combination with the object category of map data, obtains image tiles corresponding to sample labels from remote sensing images according to the requirements of a deep learning platform, and generates sample data based on the image tiles; a model training service, which is used to receive sample data, perform model training using a deep learning platform, and generate an optimal prediction model; an object classification service, which is used to obtain object classification results from remote sensing images according to the optimal prediction model; and a post-classification processing service, which is used to perform subsequent processing on the object classification results. The present invention adopts a service containerized technical architecture, which reduces the degree of system coupling, facilitates system upgrades, migration deployment, and makes the engineering application of automatic interpretation of objects in remote sensing images easy to implement.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing technology, and in particular to a system and method for automatic interpretation of remote sensing imagery objects based on deep learning. Background Art

[0002] The means of obtaining ground object information from remote sensing images include manual interpretation and computer automatic classification. Manual interpretation has the characteristics of slow interpretation speed and high accuracy, and is the main means of engineering application. Computer automatic classification has the characteristics of fast interpretation speed and low accuracy, and is an auxiliary means of engineering application. With the continuous development of earth observation technology, remote sensing data has increased dramatically, and the automatic classification of ground objects using remote sensing images has gradually moved from theoretical research to engineering application.

[0003] With the theoretical breakthrough of deep learning technology, artificial intelligence has achieved remarkable results. The use of deep learning for remote sensing image object classification has gradually become a research hotspot. Compared with traditional automatic classification technology, deep learning has the following characteristics: First, the classification accuracy is high, and the classification accuracy of a single object can reach up to 95%; second, the jagged feeling of the object boundary in the classification result is not obvious, and the local area is closer to the distribution law of the object; third, deep learning automatically performs feature learning, and there is no need to manually screen feature bands as the classification data source.

[0004] However, in the engineering application of automatic classification of remote sensing image objects, the use of deep learning technology has the following problems: First, model training requires a large number of samples, and sample collection is time-consuming and labor-intensive; second, model training requires a high-performance GPU; it relies heavily on the software environment and is difficult to deploy and migrate; third, model trainers need to have more deep learning theoretical knowledge and be able to adjust training parameters in a timely manner based on system feedback; the process from model prediction to platform application is complex, and requires high professional capabilities of personnel. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.

[0006] To this end, according to a first aspect of the present invention, a remote sensing image automatic interpretation system is provided, which is characterized by comprising:

[0007] The sample automatic collection service is used to randomly collect sample label data in combination with the object category attributes of the map data, obtain the image tiles corresponding to the sample labels from the remote sensing images according to the requirements of the deep learning platform, and generate sample data based on the image tiles;

[0008] A model training service, which is used to receive the sample data, perform model training using a deep learning platform, and generate an optimal prediction model;

[0009] A land object classification service, used to obtain land object classification results from remote sensing images according to the optimal prediction model;

[0010] The post-classification processing service is used to perform subsequent processing on the classification results of the above-mentioned objects.

[0011] Furthermore, the sample automatic collection service collects sample label data based on open source maps and map tile technology, and collects image tiles corresponding to sample labels from remote sensing images.

[0012] Furthermore, the sample data formats required by the deep learning platform include true color images, object category description files, remote sensing image tiles, and training verification statistics files.

[0013] Furthermore, the fully convolutional neural network (FCNN) algorithm is integrated into the deep learning platform in the model training service.

[0014] Furthermore, the subsequent processing includes vectorization conversion of the object classification results and integration and display with the GIS platform.

[0015] According to a second aspect of the present invention, there is provided a method for automatic interpretation of objects in remote sensing images, characterized in that it is executed by the system according to the first aspect.

[0016] Furthermore, the sample automated collection step includes:

[0017] Clip vector map tiles of areas of interest from open source maps;

[0018] According to the random algorithm, M map vector tiles are selected in the region of interest and a tile description file is generated;

[0019] Based on the feature attribute information of the map vector tile, generate the true color image and grayscale image corresponding to each tile;

[0020] Referring to the tile description file, extracting the image tile corresponding to each vector tile from the remote sensing image;

[0021] Delete files with the same name that have inconsistent ground features between remote sensing images and true color images;

[0022] All sample data sets are allocated to training, validation, and testing stages according to predetermined proportions and stored in training and validation statistical files.

[0023] Furthermore, the model training step includes:

[0024] Receiving the selection of a pre-trained model, sample data, and the input of the number of iterations and step size parameters, and executing model training by a deep learning platform, wherein the deep learning platform integrates a fully convolutional neural network (FCNN) algorithm;

[0025] The parameters are continuously adjusted according to the feedback results during the training process. When the classification accuracy exceeds the threshold, the training is completed and the optimal prediction model is generated.

[0026] Furthermore, the ground feature classification step includes:

[0027] Perform tile processing on a remote sensing image to generate remote sensing image tiles;

[0028] Based on the remote sensing image tiles, the optimal prediction model is used to derive and obtain multiple ground object classification sub-data;

[0029] Multiple object classification sub-data are stitched together to generate a complete picture as the object classification result.

[0030] Furthermore, the post-classification processing step includes:

[0031] Vectorizing the ground object classification result to generate vectorized ground object classification data;

[0032] Eliminating fragmented polygons in the vectorized feature classification data to generate merged vector data;

[0033] The merged vector data is superimposed on the latest remote sensing image map, and the ground feature data that does not match the latest remote sensing image is modified using a vector editing tool.

[0034] The modified vector data is published as standard tiles and published on a GIS platform for visual display.

[0035] The present invention adopts a service containerized technical architecture to reduce the degree of system coupling and facilitate system upgrades, migration and deployment; combines open source maps and tile technology to realize the automation of sample collection, providing a data basis for large-scale deep learning training; through containerized packaging of services, it solves the problem of difficulty in software migration and deployment; through service packaging such as object classification and classification post-processing, it solves the process processing problem from classification prediction to platform application, thereby making it easy to realize the engineering application of automatic interpretation of remote sensing image objects.

[0036] Additional aspects and advantages of the present invention will be given in part in the following description and, in part, will be obvious from the following description or learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0038] Figure 1 is a structural diagram of a remote sensing image and ground object automatic interpretation system according to the present invention;

[0039] Figure 2 is a flow chart of a method for automatic interpretation of ground objects in remote sensing images according to the present invention;

[0040] Figure 3 It is a schematic diagram of the automatic interpretation process of remote sensing image objects according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0042] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0043] refer to Figure 1 , which shows a remote sensing image automatic interpretation system according to the present invention. The present invention adopts a service-oriented technical architecture, encapsulates services in containers, and generates sample automatic collection services 11, model training services 12, object classification services 13 and classification post-processing services 14.

[0044] The sample automatic collection service 11 randomly collects sample label data in combination with the object category attributes of the map data, obtains image tiles corresponding to the sample labels from the remote sensing images according to the requirements of the deep learning platform, and generates sample data based on the image tiles;

[0045] Furthermore, the sample automatic collection service 11 generates training sample data based on open source maps and map tile technology, and the sample data content includes true color images, ground feature category description files, remote sensing image tiles and training verification statistical files.

[0046] The model training service 12 is used to receive sample data, perform model training using a deep learning platform, and generate an optimal prediction model;

[0047] Furthermore, in the model training service 12, the fully convolutional neural network (FCNN) algorithm is integrated with a deep learning platform, and the deep learning platform is, for example, a Caffe platform.

[0048] The object classification service 13 is used to obtain object classification results from remote sensing images according to the optimal prediction model;

[0049] The post-classification processing service 14 is used to perform subsequent processing on the ground feature classification results.

[0050] Furthermore, the subsequent processing includes vectorization conversion of the object classification results and integration and display with the GIS platform.

[0051] refer to Figure 2 , which shows a remote sensing image automatic interpretation method according to the present invention, the method is based on reference Figure 1 The system is executed, and the method includes:

[0052] S21, sample automatic collection step: randomly select several areas on the map, and generate sample data according to the sample data format required by the deep learning platform;

[0053] Specifically, the step S21 includes:

[0054] Clip vector map tiles of areas of interest from open source maps;

[0055] According to the random algorithm, M map vector tiles are selected in the region of interest, and a tile description file is generated; M is optionally 50,000.

[0056] Based on the feature attribute information of the map vector tile, generate the true color image and grayscale image corresponding to each tile;

[0057] Referring to the tile description file, extracting the image tile corresponding to each vector tile from the remote sensing image;

[0058] Delete files with the same name that have inconsistent ground features between remote sensing images and true color images;

[0059] All sample data sets are allocated to training, validation, and testing stages according to predetermined proportions and stored in training and validation statistical files.

[0060] S22, model training step: receiving sample data, using the deep learning platform to perform model training and generate an optimal prediction model;

[0061] Specifically, the step S22 includes:

[0062] Receiving the selection of a pre-trained model, sample data, and the input of the number of iterations and step size parameters, and executing model training by a deep learning platform, wherein the deep learning platform integrates a fully convolutional neural network (FCNN) algorithm;

[0063] The parameters are continuously adjusted according to the feedback results during the training process. When the classification accuracy exceeds the threshold, the training is completed and the optimal prediction model is generated.

[0064] S23, ground object classification step: obtaining ground object classification results from the remote sensing image according to the optimal prediction model;

[0065] Specifically, the step S23 includes:

[0066] Perform tile processing on a remote sensing image to generate remote sensing image tiles;

[0067] Based on the remote sensing image tiles, the optimal prediction model is used to derive and obtain multiple ground object classification sub-data;

[0068] Multiple object classification sub-data are stitched together to generate a complete picture as the object classification result.

[0069] S24, post-classification processing step: performing subsequent processing on the ground feature classification results.

[0070] Specifically, the step S24 includes:

[0071] Vectorizing the ground object classification result to generate vectorized ground object classification data;

[0072] Eliminating fragmented polygons in the vectorized feature classification data to generate merged vector data;

[0073] The merged vector data is superimposed on the latest remote sensing image map, and the ground feature data that does not match the latest remote sensing image is modified using a vector editing tool and then displayed.

[0074] refer to Figure 3 , which shows the automatic interpretation process of remote sensing image objects according to an embodiment of the present invention. The remote sensing image automatic interpretation system based on deep learning adopts a service containerized technical architecture to reduce the degree of system coupling and facilitate system upgrades, migration and deployment. The processing process includes four parts: automated collection, model training, object classification, and post-classification processing. Each part is a containerized service.

[0075] Automatic sample collection is encapsulated as a brain-data service, which randomly selects several areas on the map based on open source maps and map tile technology. Each area is a standard 512*512 pixel map tile, which is stored as a rasterized image and description file. Model training integrates the fcnn (full convolutional neural network) algorithm based on the caffe platform (a deep learning platform) and is encapsulated as a brain-train service. Users input sample data, fill in training parameters for model training, and continuously adjust parameters based on feedback results to generate the optimal prediction model. Land object classification is encapsulated as a brain-run service. Users input original remote sensing images and the optimal prediction model. The service tiles the original image data and performs classification processing to obtain the land object classification results. Post-classification processing is encapsulated as a brain-mapping service, which is mainly responsible for vectorization conversion of land object classification results, integration with the GIS platform, and display.

[0076] Among them, the brain-data service realizes automatic sample collection, including:

[0077] Download global vector data from open source maps and input the region range as the area of ​​interest for sample collection;

[0078] Set the map tile collection rules, specify the types of objects to be collected, and optionally, the tile size is 512*512 pixels or 256*256 pixels. Try to keep 1-3 types of objects in each collection. Randomly select 50,000 map vector tiles in the area of ​​interest, automatically complete the collection according to the user configuration, and generate a tile description file to describe the tile information of the collected samples.

[0079] Based on the map vector tiles, the true color image and grayscale image corresponding to each tile are generated as the labels of the training samples. The distribution of objects can be clearly seen in the true color image, while the sample input of the deep convolutional neural network must be a grayscale image;

[0080] Referring to the tile description file, extracting the image tile corresponding to each vector tile from the original remote sensing image or the remote sensing image map obtained by tiling the remote sensing image;

[0081] Receive operation instructions. For true color images and remote sensing image tiles with the same file name, delete files with inconsistent ground objects in the same file name to avoid the influence of incorrect training sample labels on model training. The true color images and remote sensing image tiles can be manually compared one by one and operation instructions can be issued;

[0082] All training sample data sets are allocated to training, validation, and testing stages in a 2:2:1 ratio and stored in a descriptive file.

[0083] The brain-train service implements model training, including:

[0084] Receive the selection of a pre-trained model (pre-trained on the COCO dataset), input sample data, number of iterations, step size and other parameters, and perform model training;

[0085] Receive parameter adjustments, which are made by users observing the curves during the training process and continuously adjusting according to feedback results; when the classification accuracy exceeds the threshold, that is, the user's psychological expectation value (the automatic classification accuracy acceptable to the user), the final machine training is completed and the optimal prediction model is generated.

[0086] The brain-run service implements object classification, including:

[0087] Input a remote sensing image to be classified, perform tile processing, and generate remote sensing image tiles with a standard tile size of 512*512 pixels;

[0088] Use the above optimal prediction model to perform object classification prediction on the generated remote sensing image tiles, and output the object classification sub-data, which is a rasterized image;

[0089] The images of multiple object classification sub-data are stitched together to generate a complete image as the object classification result.

[0090] The brain-mapping service implements post-classification processing, including:

[0091] Vectorize the complete picture of the object classification result to generate vectorized object classification data;

[0092] Set the minimum polygon threshold, merge polygons with insufficient area into surrounding polygons, remove fragmented polygons, and generate merged vector data;

[0093] After overlaying the vector data with the latest remote sensing image map, the user uses the vector editing tool to modify and adjust the areas where the ground object classification results do not match the remote sensing image, and display them;

[0094] In addition, the brain-mapping service can randomly collect some automatically classified vector data and compare them with the open source vector data to obtain the final classification accuracy of the automatic classification.

[0095] The modified vector data is published as standard tiles and published on a GIS platform for visual display.

[0096] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For a person skilled in the art, the present invention can have various changes and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A remote sensing image automatic interpretation system, characterized in that: It includes multiple containerized services, namely: The sample automatic collection service is used to randomly collect sample label data in combination with the object category attributes of the map data, obtain the image tiles corresponding to the sample labels from the remote sensing images according to the requirements of the deep learning platform, and generate sample data based on the image tiles; the sample data formats required by the deep learning platform include true color images, object category description files, remote sensing image tiles, and training verification statistics files; Model training service, which is used to receive sample data, use the deep learning platform to train the model and generate the optimal prediction model; A land object classification service, used to obtain land object classification results from remote sensing images according to the optimal prediction model; Post-classification processing service, used for subsequent processing of the object classification results; The automated sample collection service is specifically used for: Clip vector map tiles of areas of interest from open source maps; According to the random algorithm, M map vector tiles are selected in the region of interest and a tile description file is generated; Based on the feature attribute information of the map vector tile, generate the true color image and grayscale image corresponding to each tile; Referring to the tile description file, extracting the image tile corresponding to each vector tile from the remote sensing image; Delete files with the same name that have inconsistent ground features between remote sensing images and true color images; All sample data sets are allocated to training, validation, and testing stages in a 2:1:1 ratio, and the allocation rules are stored in the training and validation statistical files.

2. The system according to claim 1, characterized in that In the model training service, the fully convolutional neural network (FCNN) algorithm is integrated into the deep learning platform.

3. The system according to claim 1, characterized in that The subsequent processing includes vectorization conversion of the object classification results and integration and display with the GIS platform.

4. A method for automatic interpretation of remote sensing image objects, characterized in that: Executed by the system according to any one of claims 1 to 3, the method comprises: Sample automatic collection step: randomly collect sample label data in combination with the object category attributes of map data, obtain image tiles corresponding to sample labels from remote sensing images according to the requirements of the deep learning platform, and generate sample data based on the image tiles; the sample data format required by the deep learning platform includes true color images, object category description files, remote sensing image tiles and training verification statistics files; Model training step: receiving sample data, using the deep learning platform to train the model and generate the optimal prediction model; Land object classification step: obtaining land object classification results from remote sensing images according to the optimal prediction model; Post-classification processing step: performing subsequent processing on the ground feature classification results; The sample automated collection step comprises: Clip vector map tiles of areas of interest from open source maps; According to the random algorithm, M map vector tiles are selected in the region of interest and a tile description file is generated; Based on the feature attribute information of the map vector tile, generate the true color image and grayscale image corresponding to each tile; Referring to the tile description file, extracting the image tile corresponding to each vector tile from the remote sensing image; Delete files with the same name but inconsistent ground features in remote sensing images and true color images; All sample data sets are allocated to training, validation, and testing stages in a 2:1:1 ratio, and the allocation rules are stored in the training and validation statistical files.

5. The method according to claim 4, characterized in that The model training step includes: Receiving the selection of a pre-trained model, sample data, and the input of the number of iterations and step size parameters, and executing model training by a deep learning platform, wherein the deep learning platform integrates a fully convolutional neural network (FCNN) algorithm; The parameters are continuously adjusted according to the feedback results during the training process. When the classification accuracy exceeds the threshold, the training is completed and the optimal prediction model is generated.

6. The method according to claim 5, characterized in that The ground feature classification step comprises: Perform tile processing on a remote sensing image to generate remote sensing image tiles; Based on the remote sensing image tiles, the optimal prediction model is used to derive and obtain multiple ground object classification sub-data; Multiple object classification sub-data are stitched together to generate a complete picture as the object classification result.

7. The method according to claim 6, characterized in that The post-classification processing steps include: Vectorizing the ground object classification result to generate vectorized ground object classification data; Eliminating fragmented polygons in the vectorized feature classification data to generate merged vector data; Overlaying the merged vector data with the latest remote sensing image map, and using a vector editing tool to modify the ground feature data that does not match the latest remote sensing image; The modified vector data is published as standard tiles and published on a GIS platform for visual display.

Images