A solid waste site recognition method, device, and storage medium based on deep learning

Abstract

The present invention relates to the technical field of environmental monitoring, and specifically relates to a solid waste site identification method, device and storage medium based on deep learning. It includes: obtaining a satellite remote sensing image of the area to be detected; inputting the satellite remote sensing image into a pre-trained deep learning network detection model to obtain a solid waste detection result of the area to be detected. In this way, the method of using a deep learning model to detect solid waste based on satellite remote sensing images eliminates the need for manual on-site inspections, saving labor costs and time costs, and at the same time improving the detection efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of environmental monitoring, and particularly relates to a solid waste site identification method, device and storage medium based on deep learning. Background Art

[0002] Solid waste dumps mainly include domestic waste, construction waste, industrial and mining deposits, etc. according to their sources. Due to the variety of solid waste dumps, different characteristics, and large differences in size and shape, the current supervision of solid waste dumps mainly relies on manual inspections, which require fixed personnel, regular and fixed-point inspections of corresponding grids. When law enforcement officers conduct daily grid inspections and find suspicious vehicles transporting solid waste within an enterprise, they initially obtain the first clue by tracking the vehicles and analyzing their routes.

[0003] The manual inspection method has high accuracy, but it depends on the experience of law enforcement officers and has a huge workload. In the process of the country increasing investment in science and technology and fully promoting the pilot construction of "waste-free cities", the areas that need to be inspected are also larger. Therefore, if the same manual inspection is adopted, it will consume a huge amount of human resources, and at the same time, it will also take a huge amount of time and have low work efficiency. Summary of the Invention

[0004] The main technical problem to be solved by the present invention is that the existing manual inspection of solid waste dumps has high costs and low efficiency.

[0005] A solid waste site identification method based on deep learning, comprising:

[0006] Obtaining a satellite remote sensing image of the area to be detected;

[0007] Inputting the satellite remote sensing image into a pre-trained deep learning network detection model to obtain a solid waste detection result of the area to be detected.

[0008] In one embodiment, the solid waste detection result includes the category of the dump, the range of the dump, and location information.

[0009] In one embodiment, the deep learning network model is trained by the following method:

[0010] Sample library establishment: Establishing a dump sample image library;

[0011] Training step: Training an initial deep learning network model with some sample images in the sample image library;

[0012] Detection step: Testing the trained initial network model with the remaining sample images in the sample image library;

[0013] Judge whether the detection accuracy of the trained initial model meets the preset requirements according to the test results. If so, use the currently trained network initial model as the network detection model; otherwise, continue to loop through the above training steps and detection steps until the detection accuracy of the trained initial model meets the preset requirements.

[0014] In one embodiment, the establishment of the sample library includes: obtaining the satellite remote sensing image of the sample yard as the sample image, and dividing the sample image into sample image blocks of 224*224 size.

[0015] In one embodiment, the establishment of the sample library further includes:

[0016] Preprocessing of the sample image: performing atmospheric correction, orthorectification, orthorectification of panchromatic data, and multi-spectral - panchromatic fusion processing on the obtained sample image to obtain image data with a meter-level spatial resolution;

[0017] Outlining the sample image; through visual interpretation and combining with the existing spatial data of the yard map, drawing the sample data of the yard and dividing the yard types on the sample image.

[0018] A solid waste yard identification device based on deep learning, comprising:

[0019] An acquisition module for obtaining the satellite remote sensing image of the area to be detected;

[0020] A detection module for inputting the satellite remote sensing image into a pre-trained deep learning network detection model to obtain the solid waste detection result of the area to be detected.

[0021] In one embodiment, the solid waste detection result includes the category of the yard, the yard range, and the location information.

[0022] In one embodiment, the deep learning network model is trained by the following method:

[0023] Establishment of the sample library: Establish a sample image library of the yard;

[0024] Training step: Use most of the sample images in the sample image library and the XML file of the yard information in the image to train the initial deep learning network model;

[0025] Detection step: Use the remaining small part of the sample images in the sample image library and the XML file of the yard information in the image to test the trained network initial model;

[0026] Judge whether the detection accuracy of the trained initial model meets the preset requirements according to the test results. If so, use the currently trained initial model of the network as the network detection model; otherwise, continue to loop through the above training steps and detection steps until the detection accuracy of the trained initial model meets the preset requirements.

[0027] In one embodiment, the establishment of the sample library includes: obtaining the satellite remote sensing image of the sample yard as the sample image, and dividing the sample image into sample image blocks of 224*224 size.

[0028] A computer-readable storage medium, on which a program is stored, and the program can be executed by a processor to implement the method as described above.

[0029] According to the solid waste field recognition method based on deep learning in the above embodiment, it includes: obtaining the satellite remote sensing image of the area to be detected; inputting the satellite remote sensing image into the pre-trained deep learning network detection model to obtain the solid waste detection result of the area to be detected. In this way, the method of using a deep learning model to detect solid waste based on satellite remote sensing images eliminates the need for on-site inspections by humans, saving labor costs and time costs, and improving the detection efficiency at the same time. Description of the Drawings

[0030] Figure 1 It is the flowchart of the solid waste field recognition method according to the embodiment of the present application;

[0031] Figure 2 It is the flowchart of the detection model training method according to the embodiment of the present application;

[0032] Figure 3 It is the schematic diagram of the VGG16 network structure according to the embodiment of the present application;

[0033] Figure 4 It is the flowchart of the model training and detection method according to the embodiment of the present application;

[0034] Figure 5a It is the remote sensing image of the steel material yard according to the embodiment of the present application;

[0035] Figure 5b It is the image of the steel material yard taken during the on-site inspection according to the embodiment of the present application;

[0036] Figure 6a It is the remote sensing image of the sand and stone material yard according to the embodiment of the present application;

[0037] Figure 6b It is the image of the sand and stone material yard taken on-site according to the embodiment of the present application;

[0038] Figure 7a It is the remote sensing image of the coal material yard according to the embodiment of the present application;

[0039] Figure 7b It is a field-shot image of the coal material yard in the embodiment of the present application;

[0040] Figure 8a It is a remote sensing image of the building material yard in the embodiment of the present application;

[0041] Figure 8b It is a field-shot image of the building material yard in the embodiment of the present application;

[0042] Figure 9 It is the yard distribution map detected by the network model in the embodiment of the present application;

[0043] Figure 10 It is a schematic structural diagram of the recognition device in the embodiment of the present application;

[0044] Figure 11 It is a schematic diagram of the pooling process in the embodiment of the present application;

[0045] Figure 12 It is a schematic diagram of the principle of anchor box generation in the embodiment of the present application;

[0046] Figure 13 It is a schematic diagram of obtaining the anchor box size in the embodiment of the present application;

[0047] Figure 14 It is a flow chart of yard recognition in the embodiment of the present application;

[0048] Figure 15 It is a schematic diagram of the determined anchor box and the real position of the yard in the embodiment of the present application. Detailed implementation manners

[0049] The present invention will be further described in detail below in conjunction with the accompanying drawings through specific implementation manners. Similar elements in different implementation manners are labeled with related similar element numbers. In the following implementation manners, many detailed descriptions are provided to enable a better understanding of the present application. However, those skilled in the art can easily recognize that some of the features can be omitted in different situations, or can be replaced by other elements, materials, or methods. In some cases, some operations related to the present application are not shown or described in the specification, which is to avoid the core part of the present application being overwhelmed by excessive descriptions. For those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations based on the descriptions in the specification and the general technical knowledge in the art.

[0050] In addition, the features, operations, or characteristics described in the specification can be combined in any appropriate manner to form various embodiments. At the same time, the steps or actions in the method description can also be reordered or adjusted in a manner obvious to those skilled in the art. Therefore, the various sequences in the specification and drawings are only for clearly describing a certain embodiment and do not mean a necessary sequence, unless it is stated that a certain sequence must be followed.

[0051] With the maturity of deep learning, various types can be extracted using deep learning models. The applicant has studied the intelligent recognition of the yard through neural networks. Through the AI model and remote sensing technology, the yard is classified and a large number of samples are selected to automatically learn the characteristics of each type of yard. The process can automatically learn and establish the extraction model of each type of yard without caring about the selection and combination of features, thus realizing the automated yard extraction analysis.

[0052] However, there is still little work on using AI models and remote sensing technology to intelligently identify solid waste yards in complex scenarios. In this application, the Pearl River Delta urban agglomeration is selected as the test area. Based on a large number of field investigations in the early stage, the yard types that cause environmental damage are selected. Secondly, a large number of samples are selected for each type of yard. Finally, the VGG16 - Frcnn neural network is used to learn the samples to obtain the extraction model of each type of yard, thus realizing the identification of yards in complex scenarios.

[0053] Embodiment 1:

[0054] Please refer to Figure 1 and Figure 3 This embodiment provides a method for identifying solid waste yards based on deep learning, which includes:

[0055] Step 101: Obtain the satellite remote sensing image of the area to be detected. In this embodiment, a high - precision satellite remote sensing device is used to collect image data with a meter - level spatial resolution.

[0056] Step 102: Input the satellite remote sensing image into a pre - trained deep - learning network detection model to obtain the solid waste detection result of the area to be detected. In this embodiment, through the network detection model, the category, range, and location information of the yard can be detected. For example, it can be detected that the yard category in the current area to be detected is a steel material yard, a sand and gravel yard, a coal yard, a building material yard, a domestic waste yard, a construction waste yard, etc. At the same time, combined with map information, the range and location information of the yard are marked on the map, such as Figure 9 , after detecting the yard locations in each area, mark them at the corresponding positions on the map, where the area marked by the square is the range of the yard.

[0057] Among them, such as Figure 2 and Figure 3, the deep learning network model of this embodiment is obtained through the following method:

[0058] Step 201: Establishment of sample library: Establish a sample image library of the yard. In this embodiment, high-resolution remote sensing image data such as GF2 and GF7 is used.

[0059] Step 202: Training step: Use most of the sample images in the sample image library to train the initial deep learning network model. For example, the initial network model in this embodiment selects the vgg16 network model, such as model1, and uses 80% of the sample images in the sample library to train model1. Through model1, the image blocks with yards can be recognized. Use the FRCNN model, such as model2, and use the sample image library and the yard distribution range in the corresponding sample images to train model2. Through the model2, further detection is performed on the yard images detected by model1 to detect the yard range in the image blocks.

[0060] Please refer to Figure 5a 、 Figure 6a 、 Figure 7a 、 Figure 8a As shown, they are respectively the ranges and positions of each yard detected by the network detection model model2.

[0061] Step 203: Detection step: Use the remaining small part of the sample images in the sample image library to test the trained initial network model. For example, use 20% of the sample images in the sample image library to test the recognition accuracy of the trained vgg16 network model model1. Use 20% of the sample images in the sample image library and the XML file of the yard distribution range in the images to test the recognition accuracy of the trained FRCNN network model model2.

[0062] Step 204: Judge whether the detection accuracy of the trained initial model meets the preset requirements according to the test results. If so, use the currently trained network initial model as the network detection model; otherwise, continue to loop the above training step and detection step until the detection accuracy of the trained initial model meets the preset requirements.

[0063] Among them, in the process of establishing the sample library in this embodiment, it includes:

[0064] 1.1. Preprocessing of sample images: Perform atmospheric correction, orthorectification, orthorectification of panchromatic data, and multispectral-panchromatic fusion processing on the obtained sample images to obtain image data with a meter-level spatial resolution.

[0065] 1.2. Outline the sample images; by visual interpretation and combining with the existing spatial data of the yard map, draw the sample data of the yard and classify the yard types on the sample images. For example, the sample images are classified into types such as steel material yards, sand and gravel yards, coal yards, building material yards, domestic waste yards, construction waste yards, etc. For each type of yard, 100 - 200 sample images are outlined.

[0066] The naming rule for the sample images is: ullon_ullat_rdlon_rdlat_wgs84_gf2.tif. The meanings of the parameters in the file name are: ulon: the longitude of the upper left corner of the sample image, ullat: the latitude of the upper left corner of the sample image, rdlon: the longitude of the lower right corner of the sample image, rdlat: the latitude of the lower right corner of the sample image, wgs84: the coordinate system of the sample image, gf2: the satellite sensor to which the sample image belongs; use an XML file to mark the basic information of the yard and the positions (distribution ranges) of the four corner points in the sample image. The XML file structure is as follows:

[0067]

[0068]

[0069] 1.3. Manually investigate and verify the yard information. According to the outlined sample positions and category information, please refer to Figure 5b , Figure 6b , Figure 7b , Figure 8b . Manually go to the site for field investigation and take pictures to verify the yard information, and correct or eliminate the incorrect yard sample information.

[0070] Among them, when obtaining the satellite remote sensing image of the sample yard as the sample image in this embodiment, the sample image is divided into sample image blocks of 224 * 224 size. Similarly, when detecting the satellite remote sensing image of the area to be detected, the image to be detected is also divided into image blocks of 224 * 224 size, and the network detection model model1 is used to screen out the image blocks suspected of having a yard from tens of millions of image blocks.

[0071] The structure of the vgg16 network model in this embodiment includes: a convolutional layer, a pooling layer, and a fully connected layer.

[0072] Among them, the convolution process of the convolutional layer is to use a convolution kernel to continuously scan the pixel matrix of each layer step by step. The value scanned each time will be multiplied by the number in the corresponding position of the convolution kernel, and then added up. The obtained value will generate a new matrix.

[0073] The pooling process of the pooling layer is as follows: The pooling operation is equivalent to a dimensionality reduction operation. There is max pooling and average pooling, and max pooling is the most commonly used. Through the pooling layer, the spatial size of the data is continuously reduced, and the number of parameters and the amount of computation will decrease accordingly, which controls overfitting to a certain extent.

[0074] The fully connected of the fully connected layer is as follows: For the n-1 layer and the n layer, any node in the n-1 layer is connected to all nodes in the n layer. That is, when each node in the n layer is calculating, the input of the activation function is the weighted sum of all nodes in the n-1 layer.

[0075] The VGG16-Frcnn neural network structure of this embodiment is as follows:

[0076] 1: Feature map extraction module: This embodiment is implemented using the part before the fully connected layer of the vgg16 neural network structure (model1). Through a series of convolution, feature activation (Relu), and pooling operations on the sample image, the feature map corresponding to the sample image is obtained. For example, Figure 11 , after the input image (224*224) is pooled, its dimension is reduced by 16, and the output feature map dimension is (18*18).

[0077] 2: Region Proposal Network RPN (Region Proposal Network): Process the feature map extracted in step 1 to obtain region proposals. This part is mainly implemented through anchor boxes.

[0078] 2.1 Convolution operation: First, perform a convolution and feature activation operation on the features in step 1.

[0079] 2.2 Anchor box generation. An anchor box is a rectangle on the image. The vertices of the rectangle record the position information, and the area of the rectangle records the range information. Each anchor box is a window with different aspect ratios and areas.

[0080] Each pixel point in the feature map will generate 9 anchor boxes according to the window sizes of (8, 16, 32) and the aspect ratios of (1:1, 1:2, 2:1). Therefore, a total of (18*18*9) anchor boxes are generated. The anchor box is represented by a vector (x, y, w, h), which respectively represent the center point coordinates, length, and width of the anchor box on the image (18*18*9*4). At the same time, each anchor box is divided into positive and negative, recording whether the anchor box contains yard information (18*18*9*2). The principle of anchor box generation is as Figure 12 shown.

[0081] 2.3 Region candidate box generation. By judging the positive and negative of the anchor boxes, it is determined whether the anchor boxes contain yard information. For the anchor boxes containing yard information, a regression operation is performed with the true position of the yard in the image (recorded in the XML file). By adjusting the offset and scaling of the anchor boxes, anchor boxes close to the true position of the yard on the ground are obtained as candidates (Proposals). As Figure 15 shown, where the border of the small dots is the range of the true distribution position of the yard in the image recorded in the XML file. The border of the large dots is the generated anchor box. By adjusting the offset and scaling, the anchor box is made to be as close as possible to the true position in the yard.

[0082] 3: Region of Interest Pooling (ROI Pooling): The proposals (candidate boxes) obtained by the linear regression model for all positive anchors have different sizes and shapes, while the fully connected layers at the end of the network for classification and re - regression require the input to be of a fixed size. To address this issue, as Figure 13 shown, model2 first downsamples proposals of different sizes (M*N) by 16 times to obtain a size of (M / 16*N / 16), and then obtains proposals of a fixed size (7*7) through pooling layer operations.

[0083] 4: Classification: Using the obtained proposal feature maps, through the fully connected layer and the softmax normalization exponent, calculate which category each proposal specifically belongs to, and output the yard category probability vector; at the same time, use the anchor box position regression again to obtain the position offset of each proposal, which is used to regress a more accurate object detection box. The technical routes of steps 2, 3, and 4 in model2 are as Figure 14 shown:

[0084] In one embodiment, during the working process, it is also necessary to continuously update the detection model for training. For example, improve the yard sample images in the sample library to increase the recognition of different types of yards and improve the recognition accuracy, without having to rebuild the feature combination model. The yard category of the area to be detected is recognized by the network model in this embodiment, and after manual verification, the recognition accuracy reaches 82%.

[0085] Embodiment 2:

[0086] This embodiment provides a solid waste site recognition device based on deep learning, as Figure 10 shown, the recognition device includes: a collection module 301 and a detection module 302.

[0087] Among them, the acquisition module 301 is used to obtain the satellite remote sensing image of the area to be detected.

[0088] The detection module 302 is used to input the satellite remote sensing image into a pre-trained deep learning network detection model to obtain the solid waste detection result of the area to be detected.

[0089] Embodiment III:

[0090] This embodiment provides a computer-readable storage medium, on which a program is stored, and the program can be executed by a processor to implement the solid waste site recognition method provided in the above Embodiment I.

[0091] Those skilled in the art can understand that all or part of the functions of the above methods can be implemented in a hardware manner or in a computer program manner. When all or part of the functions in the above embodiments are implemented in a computer program manner, the program can be stored in a computer-readable storage medium, and the storage medium may include: read-only memory, random access memory, magnetic disk, optical disk, hard disk, etc. The above functions are implemented by a computer executing the program. For example, the program is stored in the memory of the device, and when the processor executes the program in the memory, the above all or part of the functions can be implemented. In addition, when all or part of the functions in the above embodiments are implemented in a computer program manner, the program can also be stored in a storage medium such as a server, another computer, magnetic disk, optical disk, flash drive or mobile hard disk, and is saved to the memory of the local device by downloading or copying, or the system of the local device is updated. When the processor executes the program in the memory, the above all or part of the functions in the above embodiments can be implemented.

[0092] The above uses specific examples to elaborate on the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art of the present invention, according to the idea of the present invention, several simple deductions, deformations or substitutions can also be made.

Claims

1. A solid waste site recognition method based on deep learning, characterized in that, it includes: Obtain the satellite remote sensing image of the area to be detected; Input the satellite remote sensing image into a pre-trained deep learning network detection model to obtain the solid waste detection result of the area to be detected; wherein, the solid waste detection result includes the yard range and location information; the yard range is obtained by training a first model with some sample images in the yard sample image library, and identifying the image blocks with yards through the first model; the second model is trained with the yard sample image library and the yard distribution range in the corresponding sample images, and the image blocks detected by the first model are detected through the second model.

2. The solid waste site recognition method based on deep learning according to claim 1, characterized in that, the solid waste detection result further includes the category of the yard.

3. The solid waste site recognition method based on deep learning according to claim 1, characterized in that, the deep learning network model is trained by the following method: Sample library establishment: Establish the yard sample image library; Training step: Train the initial deep learning network model with some sample images in the sample image library; wherein, the initial deep learning network model includes the first model and the second model; Detection step: Test the trained network initial model with the remaining part of the sample images in the sample image library; Judge whether the detection accuracy of the trained initial model meets the preset requirements according to the test results. If so, use the currently trained network initial model as the network detection model; otherwise, continue to loop the above training steps and detection steps until the detection accuracy of the trained initial model meets the preset requirements.

4. The solid waste site recognition method based on deep learning according to claim 3, characterized in that, the sample library establishment includes: Obtain the satellite remote sensing image of the sample yard as the sample image, and divide the sample image into sample image blocks of 224*224 size.

5. The solid waste site recognition method based on deep learning according to claim 4, characterized in that, the sample library establishment further includes: Sample image preprocessing: Perform atmospheric correction, orthorectification, orthorectification of panchromatic data, and multispectral-panchromatic fusion processing on the obtained sample image to obtain image data with a meter-level spatial resolution; Draw on the sample image; Through visual interpretation and combined with the existing yard map spatial data, draw the sample data of the yard and divide the yard types on the sample image.

6. A solid waste site recognition device based on deep learning, characterized in that, it includes: An acquisition module for obtaining the satellite remote sensing image of the area to be detected; A detection module, which is used to input the satellite remote sensing image into a pre-trained deep learning network detection model to obtain the solid waste detection result of the area to be detected; wherein, the solid waste detection result includes the yard range and location information; the yard range is obtained by training a first model with some sample images in the yard sample image library, and identifying the image blocks with yards through the first model; and training a second model with the yard sample image library and the yard distribution ranges in the corresponding sample images, and detecting the image blocks detected by the first model through the second model.

7. The solid waste yard recognition device based on deep learning according to claim 6, characterized in that the solid waste detection result further includes the category of the yard.

8. The solid waste yard recognition device based on deep learning according to claim 7, characterized in that the deep learning network model is obtained by the following method: Sample library establishment: Establish the yard sample image library; Training step: Train the initial deep learning network model with most of the sample images in the sample image library; Detection step: Test the trained initial network model with the remaining small part of the sample images in the sample image library; wherein, the initial deep learning network model includes the first model and the second model; Judge whether the detection accuracy of the trained initial model meets the preset requirements according to the test results. If so, use the currently trained network initial model as the network detection model; otherwise, continue to loop the above training step and detection step until the detection accuracy of the trained initial model meets the preset requirements.

9. The solid waste yard recognition device based on deep learning according to claim 8, characterized in that the sample library establishment includes: obtaining the satellite remote sensing image of the sample yard as the sample image, and dividing the sample image into sample image blocks of 224*224 size.

10. A computer-readable storage medium, characterized in that a program is stored on the medium, and the program can be executed by a processor to implement the method according to any one of claims 1-5.

Images