Solid waste storage yard management method based on multi-source remote sensing data and digital earth platform

Abstract

The invention discloses a solid waste storage yard management method based on multi-source remote sensing data and a digital earth platform, and the method comprises the steps: obtaining a disclosed high-resolution remote sensing solid waste storage yard data set and a multi-source satellite image, and carrying out the preprocessing operation; constructing a solid waste storage yard remote sensing target detection model, and performing fine tuning based on a remote sensing pre-training model and the preprocessed high-resolution remote sensing solid waste storage yard detection data set to obtain iterative initial model parameters; reasoning a multi-source satellite image by adopting an initial iteration model, and generating high-quality tag data by combining automatic tag generation and expert verification operation. Merging newly generated label data and the preprocessed high-resolution remote sensing solid waste storage yard data set, and repeating model training, image reasoning, automatic label generation, expert verification and new data generation to obtain an enhanced model parameter and an amplified solid waste storage yard data set; and inputting the solid waste storage yard detected from the multi-source satellite image into a visual digital earth platform to realize data storage, data query, data updating and spatial analysis, thereby completing real-time and efficient management of the solid waste storage yard. According to the invention, intelligent, refined and efficient management of the solid waste storage yard can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a solid waste dump management method based on multi-source remote sensing data and a digital earth platform. Background Art

[0002] At present, the amount of solid waste is increasing rapidly, a trend caused by rapid urbanization and unprecedented population growth. A large amount of solid waste is randomly piled up or simply landfilled, causing leachate to pollute the groundwater environment and pose a potential threat to water quality safety; at the same time, harmful gases generated during garbage dumping and accumulation permeate the air, seriously endangering the health of surrounding residents.

[0003] Remote sensing images cover a wide area and their data are updated in a timely manner, showing great application potential in urban solid waste monitoring and management. In recent years, the development of high-resolution satellite images has provided high-precision data support, making it possible to accurately capture the characteristics of solid waste dumps (color, texture, shape, etc.) in complex and changing natural and man-made backgrounds. The development of deep learning models and technological advances can realize automatic feature extraction, but it is a data-driven method, and large-scale solid waste dump remote sensing data sets are still relatively scarce. Summary of the invention

[0004] The purpose of the present invention is to propose a solid waste dump management method based on multi-source remote sensing data and a digital earth platform, so as to solve the problems of difficulty in locating solid waste dumps in remote sensing images and high cost of manual marking, and to realize intelligent, refined and efficient management of solid waste dumps.

[0005] The technical solution to achieve the purpose of the present invention is: a solid waste dump management method based on multi-source remote sensing data and digital earth platform, comprising the following steps:

[0006] Step 1: Obtain public high-resolution remote sensing solid waste dump data sets and multi-source satellite images and perform preprocessing operations;

[0007] Step 2: construct a solid waste dump remote sensing target detection model, and fine-tune it based on the remote sensing pre-trained model and the pre-processed high-resolution remote sensing solid waste dump detection dataset to obtain iterative initial model parameters;

[0008] Step 3: Use the initial iterative model to infer multi-source satellite images, combine automatic label generation and expert verification operations, and generate high-quality labeled data. Merge the newly generated labeled data with the preprocessed high-resolution remote sensing solid waste landfill dataset, repeat model training, image inference, automatic label generation, expert verification, and new data generation, and obtain enhanced model parameters and an expanded solid waste landfill dataset;

[0009] In step 4, the solid waste landfills detected from multi-source satellite images are entered into the visual digital earth platform to realize data storage, data query, data update and spatial analysis, thereby completing real-time and efficient management of the solid waste landfills.

[0010] Further, in step 1, a public high-resolution remote sensing solid waste dump data set and multi-source satellite images are obtained and preprocessed, wherein:

[0011] Obtain public high-resolution remote sensing solid waste dump target detection datasets, including Dumpsites and SWAD datasets, from AGEAOrtophotos, WorldView-3, Google Earth, and SPOT satellites, with a spatial resolution of 0.2m-1.8m; preprocessing operations on the solid waste dump dataset include label format conversion, image cropping, and label update;

[0012] The label information of the Dumpsites dataset is stored in an XML file, which contains image information, the location and category of the label box. The image information and label box location information in the XML file are extracted using a script, organized according to the format of the COCO dataset, and saved as a JSON file.

[0013] The SWAD dataset itself adopts the organizational form of COCO, so there is no need to convert the label format, but image cropping and label updating are required. After the image cropping is completed, the newly generated image is renamed, and the corresponding label box is updated according to the position of the target in the new image;

[0014] Further, in step 1, a public high-resolution remote sensing solid waste dump data set and multi-source satellite images are obtained and preprocessed, wherein:

[0015] Multi-source satellite images were obtained, including Google satellite images and domestic satellite images, with a spatial resolution of about 0.5m. To keep consistent with the images in the high-resolution remote sensing solid waste landfill dataset, only the three visible light bands of blue, green, and red were extracted, and the large image was cropped into slices of 1000×1000 pixels without setting the overlap. The pixel values ​​were then normalized to (0, 255) using the minimum-maximum stretching method.

[0016] Furthermore, in step 2, a solid waste dump remote sensing target detection model is constructed, and fine-tuned based on the remote sensing pre-trained model and the pre-processed high-resolution remote sensing solid waste dump detection dataset to obtain iterative initial model parameters. The specific method is as follows:

[0017] A solid waste dump remote sensing target detection model was constructed. After inputting the training image, ResNet-50 was used as the backbone network to extract multi-layer features. The backbone network is divided into 5 layers. The starting layer contains a 7×7 convolution layer, a batch normalization layer, a ReLU activation function layer, and a 3×3 maximum pooling layer. Layers 2-5 are four residual block groups, with 3, 4, 6, and 3 residual blocks respectively. Four feature maps C2, C3, C4, and C5 are extracted through the four-layer residual convolution network. The resolution of the feature map gradually decreases, and the feature dimension gradually increases.

[0018] Then, C2, C3, C4 and C5 are input into the feature pyramid network (FPN). Starting from C5, they are upsampled layer by layer to C4, C3 and C2. The low-level feature maps of each layer are aligned with the number of channels through 1×1 convolution and added to the upsampled high-level feature maps to complete feature fusion and construct multi-scale pyramid feature maps P2, P3, P4 and P5.

[0019] The region proposal network (RPN) is used to generate candidate regions (Proposal) on the multi-scale feature maps P2, P3, P4, and P5, and features are extracted through 3×3 convolution and the target scores and regression parameters of the candidate regions are predicted; the generated Proposal is mapped to a specific feature layer P i (i=2,3,4,5), get the region of interest R of different sizes i (i=2,3,4,5). Then, R2, R3, R4, and R5 are aligned through the RoIAlign operation and adjusted to a fixed size of 7×7. Finally, these 7×7 fixed-size feature maps are input into the detection head, and features are extracted through the fully connected layer. The final detection category and target box position are output through the classification branch and the bounding box regression branch.

[0020] The preprocessed high-resolution remote sensing solid waste landfill dataset is divided into training set and validation set in a ratio of 8:2. The Backbone weight parameters of the CMID remote sensing pre-training model are loaded into ResNet50 to train the solid waste landfill remote sensing target detection model and obtain the iterative initial model parameters.

[0021] Furthermore, a stochastic gradient descent optimizer was used in training, with momentum set to 0.9, weight decay to 0.0001, initial learning rate set to 0.02, and a multi-step learning rate decay strategy. The learning rate adjustment interval was set to [16, 22], and the learning rate decay coefficient was set to 0.1. After training for 50 epochs, the model parameters with the highest verification accuracy were retained as the initial iterative model parameters M0.

[0022] Furthermore, in step 3, the initial iterative model is used to infer multi-source satellite images, combined with automatic label generation and expert verification operations to generate high-quality labeled data. The newly generated labeled data is merged with the pre-processed high-resolution remote sensing solid waste landfill dataset, and model training, image inference, automatic label generation, expert verification and new data generation are repeated to obtain enhanced model parameters and an expanded solid waste landfill dataset. The specific method is as follows:

[0023] (1) Perform multi-view reasoning on multi-source satellite images to generate a large number of potential samples;

[0024] Assume that the original multi-source satellite image set is I sum , where one image is Img, first perform brightness transformation, sharpness transformation and contrast transformation to obtain the brightness transformation image Img b , sharpness transformation image Img s and contrast transformed image Img c ; Then apply the trained detection model to Img, Img b , Img s and Img c The target frame positions and corresponding confidence levels of the four images are obtained by inference. The original image set in which the solid waste dump target frame is not detected in the four images is called I false , and the original image set where the target box is detected is called I true , to I true Img and corresponding Img b , Img s and Img c The solid waste dump target frames of the four images are merged by taking the union set, and the intersecting target frames are eliminated by non-maximum suppression, and finally the merged target frame Bbox is obtained. merge And the corresponding confidence, Bbox merge The set of true ;

[0025] (2) Combining adaptive threshold method and expert verification to ensure label accuracy

[0026] According to B true The confidence of the image is plotted with the horizontal axis representing the confidence and the vertical axis reflecting the frequency distribution of the histogram. Then, the target frames with a confidence lower than 0.5 are removed and the remaining target frames are divided into five uniform intervals according to the confidence. Next, the average value of the lowest confidence interval of the histogram is determined as the adaptive threshold and the image is removed according to this threshold. If the confidence of all the target frames in the image is lower than the adaptive threshold, the image and the corresponding target frame are removed and the remaining images are manually marked. Thus, the remote sensing image I is screened out through adaptive threshold screening and expert verification. newand the corresponding solid waste dump target frame B new ;

[0027] (3) Iterate the above (1) and (2)

[0028] The above-screened I new and B new As new solid waste dump data, it is added to the high-resolution remote sensing solid waste dump data set with the current detection model parameters M n , n=0,1,2,3…as the basis to restart the model training and get the new model parameters M n+1 , the image set I without the detected solid waste dump target false As the input of image reasoning, the adaptive threshold screening and expert verification process is then continued, iterating until the number of images above the threshold is reduced to the set threshold.

[0029] Furthermore, in step 4, the solid waste dumps detected from the multi-source satellite images are entered into the visual digital earth platform to realize data storage, data query, data update and spatial analysis, thereby completing the real-time and efficient management of the solid waste dumps. The specific method is as follows:

[0030] Data storage includes remote sensing image information storage and solid waste dump information storage; data query includes geographic range query, area size query, category query and time query; data update is regular monitoring and data correction; spatial analysis combines population data and traffic data to evaluate the impact of solid waste dumps on the surrounding environment and the rationality of their location in regional development.

[0031] The solid waste landfill management system based on multi-source remote sensing data and digital earth platform implements the solid waste landfill management method based on multi-source remote sensing data and digital earth platform, realizes the solid waste landfill management based on multi-source remote sensing data and digital earth platform, and executes steps 1 to 4 respectively in four modules.

[0032] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the solid waste landfill management method based on multi-source remote sensing data and a digital earth platform is implemented to realize solid waste landfill management based on multi-source remote sensing data and a digital earth platform.

[0033] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the solid waste landfill management method based on multi-source remote sensing data and a digital earth platform, thereby realizing solid waste landfill management based on multi-source remote sensing data and a digital earth platform.

[0034] Compared with the existing technology, the present invention has the following significant advantages: (1) It can effectively reduce the time and manpower and material costs of remote sensing solid waste dump detection. By using limited public data sets, combined with model reasoning and expert verification, potential targets in the open world can be efficiently and accurately mined, which greatly reduces the difficulty of manual labeling in the deep learning training process. The entire data preparation and detection cycle can be shortened to one percent or even more of the original.

[0035] (2) It can effectively improve the accuracy of remote sensing solid waste dump detection. More solid waste dump samples from satellite images are introduced through human-computer interaction methods, and these new samples are allowed to participate in the iterative training process of the model. As the number of iterations increases, the total number of images participating in the training increases, which greatly improves the performance of the detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flowchart for managing solid waste dumps based on multi-source remote sensing data and the Digital Earth platform.

[0037] Figure 2 It is the detection result diagram of solid waste dump with different iteration models. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0039] The present invention provides a solid waste landfill management method based on multi-source remote sensing data and the Digital Earth platform, which integrates public data sets and multi-source satellite images, proposes an iterative training framework that integrates intelligent algorithms and expert verification, and builds a solid waste landfill management system based on the Digital Earth platform. Figure 1 As shown, the specific steps are as follows:

[0040] Step 1: Obtain public high-resolution remote sensing solid waste dump datasets and multi-source satellite images and perform preprocessing operations, including:

[0041] The publicly available high-resolution remote sensing solid waste dump datasets in China include Dumpsites and SWAD datasets. The data sources are AGEAOrtophotos, WorldView-3, Google Earth and SPOT satellites, with a spatial resolution of 0.2m-1.8m. The preprocessing operations for the solid waste dump dataset include label format conversion, image cropping and label update. The label information of the Dumpsites dataset is stored in an XML file, which contains image information, the location and category of the label box and other related information. The image information and label box location information in the XML file are extracted using a script, organized in the format of the COCO dataset, and saved as a JSON file. The SWAD dataset itself adopts the organization form of COCO, so there is no need to convert the label format. However, due to the inconsistency of the image size in this dataset, it is necessary to crop the image and update the label. After the image cropping is completed, the newly generated image is renamed, and the corresponding label is updated according to the location of the target in the new image.

[0042] The multi-source satellite images obtained include Google satellite images and domestic satellite images, with a spatial resolution of about 0.5m. To be consistent with the images in the public high-resolution remote sensing solid waste dump dataset, only the three visible light bands of blue, green and red are extracted. The large image is cropped into slices of 1000×1000 pixels, without setting the overlap, and then the pixel values ​​are normalized to (0, 255) by the minimum-maximum stretching method.

[0043] The preprocessed high-resolution remote sensing solid waste dump dataset will be used for the training and verification of the solid waste dump detection model. The remote sensing image will be used as the input data for the inference phase of the solid waste dump detection model to infer the prediction results of the solid waste dump in the remote sensing image, including the target box and confidence. The reliable part of the prediction result will be made into a label and added to the subsequent iterative training of the solid waste dump detection model together with the remote sensing image.

[0044] Step 2: Build a solid waste dump detection model, and fine-tune it based on the remote sensing pre-trained model and the pre-processed high-resolution remote sensing solid waste dump detection dataset to obtain the iterative initial model parameters, specifically:

[0045] The preprocessed high-resolution remote sensing solid waste dump dataset is divided into a training set and a validation set in a ratio of 8:2. After inputting the training image, ResNet-50 is used as the backbone network to extract multi-layer features. The backbone network is divided into 5 layers. The starting layer contains a 7×7 convolution layer, a batch normalization layer, a ReLU activation function layer, and a 3×3 maximum pooling layer. Layers 2-5 are four residual block groups, with [3, 4, 6, 3] residual blocks respectively. Four feature maps C2, C3, C4, and C5 are extracted through a four-layer residual convolutional network. The resolution of the feature map gradually decreases, and the feature dimension gradually increases (the size of C2 is 1 / 4 of the input image, and C5 is 1 / 32 of the input image);

[0046] Then, C2, C3, C4 and C5 are input into the feature pyramid network (FPN). Starting from C5, they are upsampled layer by layer to C4, C3 and C2. The low-level feature maps (C4, C3, C2) of each layer are aligned with the number of channels through 1×1 convolution and added to the upsampled high-level feature maps to complete feature fusion and construct multi-scale pyramid feature maps (P2, P3, P4, P5) to enhance the expression ability of objects of different scales.

[0047] The region proposal network (RPN) is used to generate candidate regions (Proposal) on the multi-scale feature maps P2, P3, P4, and P5, and features are extracted through 3×3 convolution and the target scores and regression parameters of the candidate regions are predicted; the generated Proposal is mapped to a specific feature layer P i (i=2,3,4,5), get the region of interest R of different sizes i (i=2,3,4,5). Then, R2, R3, R4, and R5 are aligned through the RoIAlign operation and uniformly adjusted to a fixed size of 7×7. Finally, these 7×7 fixed-size feature maps are input into the detection head, and features are extracted through the fully connected layer. The final detection category and target box position are output through the classification branch and the bounding box regression branch. The entire model combines Faster R-CNN and FPN to achieve high accuracy and efficiency in multi-scale solid waste landfill detection.

[0048] It is worth noting that during actual training, the Backbone weight parameters of the CMID remote sensing pre-training model are loaded into ResNet50 in order to enhance the feature learning ability of the target detection model for remote sensing images. CMID is a remote sensing pre-training framework based on self-supervision methods proposed in recent studies. The trained Backbone weight parameters have been open sourced and can be directly downloaded for various remote sensing downstream tasks, such as remote sensing detection of solid waste landfills.

[0049] The stochastic gradient descent optimizer was used in the training, with momentum set to 0.9, weight decay to 0.0001, initial learning rate set to 0.02, and a multi-step learning rate decay strategy, with the learning rate adjustment interval set to [16, 22] and the learning rate decay coefficient set to 0.1. After training for 50 epochs, the model parameters with the highest verification accuracy were retained as the initial iteration model parameters M0.

[0050] Step 3: Use the initial iterative model to infer multi-source satellite images, combine automatic label generation and expert verification operations, and generate high-quality labeled data. Merge the newly generated labeled data with the preprocessed high-resolution remote sensing solid waste landfill dataset, repeat model training, image inference, automatic label generation, expert verification and new data generation until the number of images above the threshold is less than 100, and stop the iteration. After each round of iteration, the enhanced model parameters and the expanded solid waste landfill dataset can be obtained. The specific operations in the iterative process are:

[0051] (1) Perform multi-view reasoning on multi-source satellite images to generate a large number of potential samples

[0052] The input of this step is the original multi-source satellite image set I sum . Take one of the images Img as an example to explain the steps. First, perform brightness transformation, sharpness transformation and contrast transformation to obtain the brightness transformation image Img b , sharpness transformation image Img s and contrast transformed image Img c , and its transformation formula is as follows:

[0053] img b =img·f b

[0054] img s =img*f s +img cov *(1-f s )

[0055] img cov =convolution(img,-1,kernel)

[0056] img c =img*f c +img con *(1-f c )

[0057]

[0058] Among them, img represents the original image; f b Represents the brightness transformation factor, f srepresents the sharpness transformation factor, f c represents the contrast transformation factor, img cov Represents the image after convolution operation, kernel is the convolution kernel size, gray2bgr is a function that converts a grayscale image to a color image. It sets each channel of the grayscale image to the same value, thereby creating a color version of the grayscale mean image. Represents an image of the same size as img, with pixel values ​​equal to the mean grayscale value.

[0059] Apply the trained solid waste dump detection model to Img, Img b , Img s and Img c Inference is performed to obtain the target frame positions and corresponding confidence levels of the four images. It is worth mentioning that since solid waste dumps account for a relatively small proportion of remote sensing images, there will be cases where the target frame of the solid waste dump is not detected in any of the four images. The original image set of this type of situation is called I false , and the original image set where the target box is detected is called I true . to I true Img and corresponding Img b , Img s and Img c The solid waste dump target frames of the four images are merged by taking the union set, and the intersecting target frames are eliminated by non-maximum suppression, and finally the merged target frame Bbox is obtained. merge And the corresponding confidence, Bbox merge The set of true .

[0060] (2) Combining adaptive threshold method and expert verification to ensure label accuracy

[0061] According to B true A histogram is drawn based on the confidence level, with the horizontal axis representing the confidence level and the vertical axis reflecting the frequency distribution of the histogram. In this process, the target boxes with a confidence level lower than 0.5 are first eliminated, and then the remaining target boxes are divided into five uniform intervals according to the confidence level. Next, the average value of the lowest confidence interval of the histogram is determined as the adaptive threshold, and the image is eliminated based on this threshold. Specifically, if the confidence levels of all target boxes in the image are lower than the adaptive threshold, the image and the corresponding target boxes are eliminated. The remaining images are then divided into four parts, and four experts are invited for preliminary verification. This stage aims to eliminate false detections caused by model misjudgment and manually mark those solid waste landfill targets that the model failed to identify. After that, an additional expert is invited to conduct a secondary screening of the results after the preliminary verification to ensure that the accuracy of the labels is further improved. So far, through the above adaptive threshold screening and expert verification, the remote sensing image I is screened out. newand the corresponding solid waste dump target frame B new .

[0062] Iterate the above (1) and (2)

[0063] The above-screened I new and B new As new solid waste dump data, it is added to the high-resolution remote sensing solid waste dump data set with the current detection model parameters M n (n=0,1,2,3…) as the basis to restart the model training and obtain the new detection model parameters M n+1 , the above-mentioned image set I of undetected solid waste dump targets false As the input of image reasoning (due to the limitation of model detection ability, I false There are still many potential solid waste landfill targets that have not been mined), and then continue to perform adaptive threshold screening and expert verification processes, iterating until the number of images above the threshold is reduced to 100.

[0064] Step 4: Solid waste dumps detected from multi-source satellite images new ,B new} Input data into the visualized digital earth platform to realize data storage, data query, data update and spatial analysis functions, thereby completing the real-time and efficient management of solid waste dumps, including:

[0065] Data storage includes remote sensing image information storage and solid waste dump information storage; data query includes geographic range query (interface interactive rectangular box, latitude and longitude range, administrative boundary), area size query, category query and time query; data update is mainly regular monitoring and data correction; spatial analysis mainly combines population data and traffic data to evaluate the impact of solid waste dumps on the surrounding environment and the rationality of their location in regional development.

[0066] Example

[0067] In order to verify the effectiveness of the scheme of the present invention, the following experiment was carried out.

[0068] Step 1: Multi-source remote sensing data acquisition and processing

[0069] The publicly available high-resolution remote sensing solid waste dump target detection datasets in China include Dumpsites and SWAD datasets. The data sources are AGEA Ortophotos, WorldView-3, Google Earth, and SPOT satellites, with a spatial resolution of 0.2m-1.8m.

[0070] The preprocessing operations of the public dataset include label format conversion, image cropping and label update. The Dumpsites dataset is organized in VOC format, and the label information is stored in an XML file, which contains information about each image, the location and category of the label box, and other related information. The script is used to extract the image information and label box location information in the XML file, remove the category information, and organize it according to the format of the COCO dataset and save it as a JSON file. On the other hand, the SWAD dataset itself adopts the organization format of COCO, so there is no need to convert the label format. However, due to the inconsistency of the image size in this dataset, it is necessary to crop the image and update the label. Specifically, in order to unify the aspect ratio, we adopted the following strategy: for 1200×600 pixel images, they were cropped into two 600×600 pixel images; for 1600×900 pixel images, a 200 pixel overlapping area was set in the center of the image, and then they were cropped into two 900×900 pixel images; for 2400×1202 pixel images, one pixel of information was cropped off from the top and bottom, and then they were cropped into two 1200×1200 pixel images. After the image cropping was completed, the newly generated images were renamed, and the corresponding labels were updated according to the position of the target in the new image. It is worth noting that during the image cropping process, if an object is cut into two objects and its width is less than 20 pixels, the label is deleted to ensure the accuracy and effectiveness of the label.

[0071] The multi-source satellite images obtained include Google satellite images and domestic satellite images, with a spatial resolution of about 0.5 m. The large-format satellite images were cropped into slices of 1000×1000 pixels without setting the overlap, and then the pixel values ​​were normalized to (0, 255) by the minimum-maximum stretching method.

[0072] The preprocessed high-resolution remote sensing solid waste dump dataset will be used for the training and verification of the solid waste dump detection model. Multiple remote sensing images will be used as input data for the model inference stage. The prediction results of the solid waste dump in the remote sensing image, including the target box and confidence level, will be obtained through detection model inference. The reliable parts of the prediction results will be made into labels and added to the subsequent iterative training of the solid waste dump detection model together with the images.

[0073] Step 2: Model building and fine-tuning

[0074] The high-resolution remote sensing solid waste dumpsite datasets Dumpsites and SWAD are divided into training sets and validation sets in a ratio of 8:2. After inputting the training image, ResNet-50 is used as the backbone network to extract multi-layer features. The backbone network is divided into 5 layers. The starting layer contains a 7×7 convolution layer, a batch normalization layer, a ReLU activation function layer, and a 3×3 maximum pooling layer. Layers 2-5 are four residual block groups, with [3, 4, 6, 3] residual blocks respectively. Four feature maps C2, C3, C4, and C5 are extracted through a four-layer residual convolutional network. The resolution of the feature map is gradually reduced, and the feature dimension is gradually increased (the size of C2 is 1 / 4 of the input image, and C5 is 1 / 32 of the input image);

[0075] Then, C2, C3, C4 and C5 are input into the feature pyramid network (FPN). Starting from C5, they are upsampled layer by layer to C4, C3 and C2. The low-level feature maps (C4, C3, C2) of each layer are aligned with the number of channels through 1×1 convolution and added to the upsampled high-level feature maps to complete feature fusion and construct multi-scale pyramid feature maps (P2, P3, P4, P5) to enhance the expression ability of objects of different scales.

[0076] The region proposal network (RPN) is used to generate candidate regions (Proposal) on the multi-scale feature maps P2, P3, P4, and P5, and features are extracted through 3×3 convolution and the target scores and regression parameters of the candidate regions are predicted; the generated Proposal is mapped to a specific feature layer P i (i=2,3,4,5), get the region of interest R of different sizes i (i=2,3,4,5). Then, R2, R3, R4, and R5 are aligned through the RoIAlign operation and uniformly adjusted to a fixed size of 7×7. Finally, these 7×7 fixed-size feature maps are input into the detection head, and features are extracted through the fully connected layer. The final detection category and target box position are output through the classification branch and the bounding box regression branch. The entire model combines Faster R-CNN and FPN to achieve high accuracy and efficiency in multi-scale solid waste landfill detection.

[0077] It is worth noting that during actual training, the Backbone weight parameters of the CMID remote sensing pre-training model are loaded into ResNet50 in order to enhance the feature learning ability of the target detection model for remote sensing images. CMID is a remote sensing pre-training framework based on self-supervision methods proposed in recent studies. The trained Backbone weight parameters are now open source and can be directly downloaded for various remote sensing downstream tasks, such as remote sensing detection of solid waste landfills.

[0078] The stochastic gradient descent optimizer is used in training, with momentum set to 0.9, weight decay to 0.0001, initial learning rate set to 0.02, and a multi-step learning rate decay strategy. The learning rate adjustment interval is set to [16, 22], and the learning rate decay coefficient is set to 0.1. Average Precision (AP) is used as the overall evaluation indicator, which can simultaneously reflect the recall rate and accuracy of the solid waste landfill detection model. AP50 refers to the area under the PR curve when the IoU threshold is 0.50. It is the most widely used indicator in the AP series of indicators. After training for 50 epochs, the model parameters with the highest verification accuracy are retained as the initial iteration model parameters M0.

[0079] Step 3: Iterative expansion of human-computer interaction

[0080] Load the initial iteration model parameter M0 to infer multi-source satellite images, combine automatic label generation and expert verification operations to generate high-quality label data. Merge the newly generated label data with the preprocessed high-resolution remote sensing solid waste landfill dataset, repeat model training, image reasoning, automatic label generation, expert verification and new data generation until the number of images above the threshold is less than 100 and stop the iteration. After each round of iteration, the enhanced model parameters and the expanded solid waste landfill dataset can be obtained. The specific operations in the iterative process are:

[0081] (1) Perform multi-view reasoning on multi-source satellite images to generate a large number of potential samples

[0082] The input of this step is the original multi-source satellite image set I sum . Take one of the images Img as an example to explain the steps. First, perform brightness transformation, sharpness transformation and contrast transformation to obtain the brightness transformation image Img b , sharpness transformation image Img s and contrast transformed image Img c .

[0083] For Img, Img b , Img s and Img c Inference is performed to obtain the target frame positions and corresponding confidence levels of the four images. It is worth mentioning that since solid waste dumps account for a relatively small proportion of remote sensing images, there will be cases where the target frame of the solid waste dump is not detected in any of the four images. The original image set of this type of situation is called I false , and the original image set where the target box is detected is called I true . to I true Img and corresponding Img b , Img s and Img cThe solid waste dump target frames of the four images are merged by taking the union set, and the intersecting target frames are eliminated by non-maximum suppression, and finally the merged target frame Bbox is obtained. merge And the corresponding confidence, Bbox merge The set of true .

[0084] (2) Combining adaptive threshold method and expert verification to ensure label accuracy

[0085] According to B true A histogram is drawn based on the confidence level, with the horizontal axis representing the confidence level and the vertical axis reflecting the frequency distribution of the histogram. In this process, the target boxes with a confidence level lower than 0.5 are first eliminated, and then the remaining target boxes are divided into five uniform intervals according to the confidence level. Next, the average value of the lowest confidence interval of the histogram is determined as the adaptive threshold τ, and the image is eliminated based on this threshold. Specifically, if the confidence level of all target boxes in the image is lower than τ, the image and the corresponding target box are eliminated. The retained images are then divided into four parts, and four experts are invited for preliminary verification. This stage aims to eliminate false detections caused by model misjudgment and manually mark those solid waste landfill targets that the model failed to identify. After that, an additional expert is invited to conduct a secondary screening of the results after the preliminary verification to ensure that the accuracy of the labels is further improved. So far, through the above adaptive threshold screening and expert verification, the remote sensing image I is screened out. new and the corresponding solid waste dump target frame B new .

[0086] (3) Iterate the above (1) and (2)

[0087] The above-screened I new and B new As new solid waste dump data, it is added to the high-resolution remote sensing solid waste dump data set with the current detection model parameters M n (n=0,1,2,3…) to restart the model training and get the new model parameters M n+1 , the above-mentioned image set I of undetected solid waste dump targets false As the input of image reasoning (due to the limitation of model detection ability, I false There are still many potential solid waste landfill targets that have not been mined), and then continue to perform the adaptive threshold screening and expert verification process, iterating until the number of images above the threshold is reduced to 100 and stopping. In this embodiment, a total of four iterations are performed.

[0088] After the first round of iterations, τ = 0.9 is taken as the threshold for generating pseudo labels. This value is the highest value of the empirical threshold in most studies. The purpose is to increase the threshold and ensure the generation of high-quality new labels.

[0089] After the second round of iteration, τ = 0.75 is taken as the threshold to generate pseudo labels. This value is calculated according to the adaptive threshold method. Because the confidence threshold is small, the data mining efficiency is low.

[0090] After the third and fourth rounds of iterations, the τ calculated by the adaptive threshold method is the same, which is 0.875.

[0091] After four rounds of iterations, a new dataset was obtained, including remote sensing image slices, true value detection frames, and geographic location information, which can be used for subsequent visualization and management of solid waste landfills. The pixel area is calculated based on the length and width of the detection frame, and the actual area is calculated based on the image resolution.

[0092] Step 4: Visualization and interaction based on the Digital Earth platform

[0093] Solid waste dumps detected from multi-source satellite images new ,B new}Enter the visual digital earth platform to realize data storage, data query, data update and spatial analysis functions:

[0094] (1) Data storage

[0095] A. Remote sensing image information storage:

[0096] The geographic coordinates, image source, resolution, and shooting time of the center point of the remote sensing slice (1000×1000 pixels) are obtained and stored in the geographic database management platform.

[0097] B. Solid waste dump information storage:

[0098] It includes the relative position (relative to the remote sensing slice), relative size (number of width pixels × number of length pixels), and absolute area (number of width pixels × resolution × number of length pixels × resolution) of the solid waste dump boundary box.

[0099] (2) Data Query

[0100] A. Geographic range query:

[0101] Interactive rectangle: Use the drawing tools in the graphical user interface to draw a rectangle directly on the map. The system will then automatically identify and query all remote sensing slices and their corresponding solid waste dump information within the area covered by the rectangle. This query method is intuitive and convenient, and is suitable for situations where the user knows the approximate area but is unsure of the exact coordinates.

[0102] Longitude and latitude range: Users can enter a specific longitude and latitude range (such as minimum longitude, maximum longitude, minimum latitude, maximum latitude), and the system will filter out all remote sensing slices and solid waste dump data within the range based on these coordinate information. This method is suitable for query scenarios that require precise geographic location information.

[0103] Administrative Boundary: Select or enter a specific administrative region (such as a country, province, city, etc.), and the system will automatically retrieve and display all remote sensing slices and solid waste dump information in the region based on the boundary data of the administrative region. This query method is particularly suitable for scenarios such as regional planning and environmental assessment that require analysis by administrative divisions.

[0104] B. Area size query:

[0105] Enter one or more area ranges (such as minimum area and maximum area), and the system will filter according to the absolute area of ​​the solid waste dump and return all the solid waste dump information that meets the area conditions. This helps to identify large landfills or small solid waste dumps.

[0106] C. Category query:

[0107] Once the user selects or enters a specific solid waste category, the system will display all the solid waste dump information belonging to that category. This query method helps to quickly locate specific types of solid waste pollution areas and provide data support for subsequent governance work.

[0108] D. Time query:

[0109] Since remote sensing slices contain shooting time information, users can specify one or more time periods, and the system will filter and display the remote sensing slices and the solid waste dump information contained in the corresponding time period according to these time periods. This is of great significance for analyzing the changing trend of solid waste dumps and evaluating the governance effect. Through time query, users can intuitively see the status changes of solid waste dumps at different time points, providing strong data support for decision-making.

[0110] (3) Data update

[0111] Regular monitoring and data correction:

[0112] Establish a regular monitoring mechanism for solid waste dumps, timely detect and handle data changes, and correct and update the data on the Digital Earth platform based on monitoring results and actual conditions. Among them, monitoring includes high-temporal resolution satellite monitoring and refined fixed-point field monitoring. The former can achieve low-cost updates, and the latter mainly corrects data for key solid waste dumps.

[0113] (4) Spatial analysis

[0114] Spatial analysis of solid waste dumps is a comprehensive process that is essential for efficient management. We collect multiple data sources and integrate them into the Digital Earth platform to assess the impact of solid waste dumps on the surrounding environment and the rationality of their location in regional development. The following is a specific analysis combining population data and traffic data:

[0115] A. Population data:

[0116] The location of solid waste dumps should be avoided as far as possible in high-density residential areas to reduce the potential impact on residents' quality of life. By analyzing the regional population density map, it is possible to identify which areas are densely populated areas, thereby guiding the site selection decision of solid waste dumps. The distance between solid waste dumps and densely populated areas is also an important indicator for assessing their risks to residents' health and the environment. A closer distance may increase the risk of air pollution, noise pollution and water pollution.

[0117] B. Traffic data:

[0118] By analyzing the traffic network diagram, it is possible to determine which roads or areas are suitable for the logistics and transportation needs of the yard, thereby optimizing resource allocation and reducing operating costs. Considering traffic flow also helps to develop emergency response plans to deal with possible emergencies such as pollutant leaks or fires.

[0119] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0120] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A solid waste dump management method based on multi-source remote sensing data and digital earth platform, characterized in that: The steps include: Step 1: Obtain public high-resolution remote sensing solid waste dump data sets and multi-source satellite images and perform preprocessing operations; Step 2: construct a solid waste dump remote sensing target detection model, and fine-tune it based on the remote sensing pre-trained model and the pre-processed high-resolution remote sensing solid waste dump detection dataset to obtain iterative initial model parameters; Step 3: Use the initial iterative model to infer multi-source satellite images, combine automatic label generation and expert verification operations, and generate high-quality labeled data. Merge the newly generated labeled data with the preprocessed high-resolution remote sensing solid waste landfill dataset, repeat model training, image inference, automatic label generation, expert verification, and new data generation, and obtain enhanced model parameters and an expanded solid waste landfill dataset; In step 4, the solid waste landfills detected from multi-source satellite images are entered into the visual digital earth platform to realize data storage, data query, data update and spatial analysis, thereby completing real-time and efficient management of the solid waste landfills.

2. The solid waste dump management method based on multi-source remote sensing data and digital earth platform according to claim 1 is characterized in that: Step 1: Obtain public high-resolution remote sensing solid waste dump datasets and multi-source satellite images and perform preprocessing operations, including: Obtain public high-resolution remote sensing solid waste dump target detection datasets, including Dumpsites and SWAD datasets, from AGEAOrtophotos, WorldView-3, Google Earth, and SPOT satellites, with a spatial resolution of 0.2m-1.8m; preprocessing operations on the solid waste dump dataset include label format conversion, image cropping, and label update; The label information of the Dumpsites dataset is stored in an XML file, which contains image information, the location and category of the label box. The image information and label box location information in the XML file are extracted using a script, organized according to the format of the COCO dataset, and saved as a JSON file. The SWAD dataset itself adopts the organizational form of COCO, and there is no need to convert the label format, but image cropping and label updating are required. After the image cropping is completed, the newly generated image is renamed, and the corresponding label box is updated according to the position of the target in the new image.

3. The solid waste dump management method based on multi-source remote sensing data and digital earth platform according to claim 1 is characterized in that: Step 1: Obtain public high-resolution remote sensing solid waste dump datasets and multi-source satellite images and perform preprocessing operations, including: Multi-source satellite images were obtained, including Google satellite images and domestic satellite images, with a spatial resolution of about 0.5m. To keep consistent with the images in the high-resolution remote sensing solid waste landfill dataset, only the three visible light bands of blue, green, and red were extracted, and the large image was cropped into slices of 1000×1000 pixels without setting the overlap. The pixel values ​​were then normalized to (0, 255) using the minimum-maximum stretching method.

4. The solid waste dump management method based on multi-source remote sensing data and digital earth platform according to claim 1 is characterized in that: Step 2: Build a remote sensing target detection model for solid waste dumps, and fine-tune it based on the remote sensing pre-trained model and the preprocessed high-resolution remote sensing solid waste dump detection dataset to obtain the iterative initial model parameters. The specific method is as follows: A solid waste dump remote sensing target detection model was constructed. After inputting the training image, ResNet-50 was used as the backbone network to extract multi-layer features. The backbone network is divided into 5 layers. The starting layer contains a 7×7 convolution layer, a batch normalization layer, a ReLU activation function layer, and a 3×3 maximum pooling layer. Layers 2-5 are four residual block groups, with 3, 4, 6, and 3 residual blocks respectively. Four feature maps C2, C3, C4, and C5 are extracted through the four-layer residual convolution network. The resolution of the feature map gradually decreases, and the feature dimension gradually increases. Then, C2, C3, C4 and C5 are input into the feature pyramid network (FPN). Starting from C5, they are upsampled layer by layer to C4, C3 and C2. The low-level feature maps of each layer are aligned with the number of channels through 1×1 convolution and added to the upsampled high-level feature maps to complete feature fusion and construct multi-scale pyramid feature maps P2, P3, P4 and P5. The region proposal network (RPN) is used to generate candidate regions (Proposal) on the multi-scale feature maps P2, P3, P4, and P5, and features are extracted through 3×3 convolution and the target scores and regression parameters of the candidate regions are predicted; the generated Proposal is mapped to a specific feature layer P i (i=2,3,4,5), get the region of interest R of different sizes i (i=2,3,4,5). Then, R2, R3, R4, and R5 are aligned through the RoIAlign operation and adjusted to a fixed size of 7×7. Finally, these 7×7 fixed-size feature maps are input into the detection head, and features are extracted through the fully connected layer. The final detection category and target box position are output through the classification branch and the bounding box regression branch. The preprocessed high-resolution remote sensing solid waste landfill dataset is divided into training set and validation set in a ratio of 8:

2. The Backbone weight parameters of the CMID remote sensing pre-training model are loaded into ResNet50 to train the solid waste landfill remote sensing target detection model and obtain the iterative initial model parameters.

5. The solid waste dump management method based on multi-source remote sensing data and digital earth platform according to claim 4 is characterized in that: The stochastic gradient descent optimizer was used in training, with the momentum set to 0.9, the weight decay to 0.0001, the initial learning rate to 0.02, and a multi-step learning rate decay strategy. The learning rate adjustment interval was set to [16, 22], and the learning rate decay coefficient was set to 0.

1. After training for 50 epochs, the model parameters with the highest verification accuracy were retained as the initial iterative model parameters M0.

6. The solid waste dump management method based on multi-source remote sensing data and digital earth platform according to claim 1 is characterized in that: Step 3: Use the initial iterative model to infer multi-source satellite images, combine automatic label generation and expert verification operations, and generate high-quality labeled data. Merge the newly generated labeled data with the preprocessed high-resolution remote sensing solid waste landfill dataset, repeat model training, image inference, automatic label generation, expert verification, and new data generation, and obtain enhanced model parameters and an expanded solid waste landfill dataset. The specific method is as follows: (1) Perform multi-view reasoning on multi-source satellite images to generate a large number of potential samples; Assume that the original multi-source satellite image set is I sum , where one image is Img, first perform brightness transformation, sharpness transformation and contrast transformation to obtain the brightness transformation image Img b , sharpness transformation image Img s and contrast transformed image Img c ; Then apply the trained detection model to Img, Img b , Img s and Img c The target frame positions and corresponding confidence levels of the four images are obtained by inference. The original image set in which the solid waste dump target frame is not detected in the four images is called I false , and the original image set where the target box is detected is called I true , to I true Img and corresponding Img b , Img s and Img c The solid waste dump target frames of the four images are merged by taking the union set, and the intersecting target frames are eliminated by non-maximum suppression, and finally the merged target frame Bbox is obtained. merge And the corresponding confidence, Bbox merge The set of true ; (2) Combining adaptive threshold method and expert verification to ensure label accuracy According to B true The confidence of the image is plotted with the horizontal axis representing the confidence and the vertical axis reflecting the frequency distribution of the histogram. Then, the target frames with a confidence lower than 0.5 are removed and the remaining target frames are divided into five uniform intervals according to the confidence. Next, the average value of the lowest confidence interval of the histogram is determined as the adaptive threshold and the image is removed according to this threshold. If the confidence of all the target frames in the image is lower than the adaptive threshold, the image and the corresponding target frame are removed and the remaining images are manually marked. Thus, the remote sensing image I is screened out through adaptive threshold screening and expert verification. new and the corresponding solid waste dump target frame B new ; (3) Iterate the above (1) and (2) The above-screened I new and B new As new solid waste dump data, it is added to the high-resolution remote sensing solid waste dump data set with the current detection model parameters M n , n=0,1,2,3…as the basis to restart the model training and get the new model parameters M n+1 , the image set I without the detected solid waste dump target false As the input of image reasoning, the adaptive threshold screening and expert verification process is then continued, iterating until the number of images above the threshold is reduced to the set threshold.

7. The solid waste dump management method based on multi-source remote sensing data and digital earth platform according to claim 1 is characterized in that: Step 4: Enter the solid waste dumps detected from multi-source satellite images into the visualized digital earth platform to achieve data storage, data query, data update and spatial analysis, thereby completing real-time and efficient management of the solid waste dumps. The specific methods are as follows: Data storage includes remote sensing image information storage and solid waste dump information storage; data query includes geographic range query, area size query, category query and time query; data update is regular monitoring and data correction; spatial analysis combines population data and traffic data to evaluate the impact of solid waste dumps on the surrounding environment and the rationality of their location in regional development.

8. A solid waste dump management system based on multi-source remote sensing data and digital earth platform, characterized in that: Implement the solid waste landfill management method based on multi-source remote sensing data and digital earth platform as described in any one of claims 1-7 to realize solid waste landfill management based on multi-source remote sensing data and digital earth platform, and perform steps 1 to 4 in four modules respectively.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the solid waste landfill management method based on multi-source remote sensing data and a digital earth platform as described in any one of claims 1 to 7 is implemented to realize solid waste landfill management based on multi-source remote sensing data and a digital earth platform.

10. A computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the solid waste landfill management method based on multi-source remote sensing data and a digital earth platform as described in any one of claims 1 to 7 is implemented to realize solid waste landfill management based on multi-source remote sensing data and a digital earth platform.

Images