An automatic recognition method for solid waste landfill remote sensing images based on deep learning

Abstract

The present invention discloses a method for automatically identifying remote sensing images of solid waste landfills based on deep learning. The method involves computer vision, collects remote sensing image data of solid waste landfills, fuses the remote sensing image data of solid waste landfills into the dataset RSD46-WHU, and constructs a new dataset RSSCD47-GU. A GSFF-VIT model improved based on the VIT model is constructed. The dataset RSSCD47-GU is trained and tested using the GSFF-VIT model and multiple deep learning models to obtain a confusion matrix. The recognition index of each model is calculated based on the confusion matrix. The models are compared and analyzed based on the confusion matrix and the recognition index to determine the effectiveness of the GSFF-VIT model. The present invention solves the problems of difficult and inefficient feature extraction during landfill identification. At the same time, the model can be quickly applied to landfill identification in other regions, thereby improving the accuracy of solid waste landfill identification.

Description

Technical Field

[0001] The present invention relates to computer vision, and more particularly, to an automatic recognition method for remote sensing images of solid waste landfills based on deep learning. Background Art

[0002] Due to the scattered nature of solid waste landfills, the sporadic occurrence of illegal landfills, and their concealed nature, landfill identification remains challenging. Currently, landfill identification methods can be broadly categorized into the following three types:

[0003] The first is the on-site survey method, which involves manual on-site visits to determine the precise location and size of landfills. This method is time-consuming and labor-intensive, with limited coverage. It often only identifies a small number of legal landfills, but fails to identify hidden illegal landfills.

[0004] The second method is to combine visual interpretation with mathematical statistical analysis, which aims to explore the spatial distribution of landfills based on the visual interpretation of remote sensing images and the basic characteristics of landfills, such as vegetation cover, spectral characteristics, and surface temperature. This method is suitable for identifying large landfills, but since illegal landfills are usually small, it is difficult to effectively identify them based solely on these remote sensing indicators. Therefore, some studies have introduced other variables, such as population density and transportation convenience, to achieve the identification of illegal landfills. However, these studies can only obtain probability maps of landfill distribution, and the identification mainly relies on external features around the landfill, rather than the characteristics of the landfill itself, and fail to achieve accurate identification of landfills.

[0005] Another approach utilizes traditional machine learning methods, where domain experts extract image features, which are then fed into a machine learning model for training and testing. This approach has achieved some success in improving recognition efficiency. However, the feature extraction process still relies on manual labor, and the quality of feature extraction directly determines the performance of the model. Furthermore, this method can easily misidentify solid waste landfills in satellite imagery as large-scale excavation areas, indicating that further improvements are needed in terms of feature differentiation and model accuracy.

[0006] In general, when carrying out landfill identification work, we must not only consider the accuracy of the identification results, but also fully consider the resource consumption during the identification process. With the increase in the number of landfills, especially the increase in the proportion of illegal landfills, the management of landfills has gradually become more difficult. If the efficiency of identification is ignored, the identification process becomes too cumbersome, time-consuming and resource-intensive, which will affect the implementation of subsequent management activities. Moreover, although existing research has provided several methods for landfill identification, these methods still require manual intervention to a large extent, resulting in a time-consuming and inefficient identification process, which is still far from the goal of achieving rapid and accurate identification. Therefore, the analysis of existing landfill identification methods and related research shows that the relevant technologies still have the following technical problems that need to be solved:

[0007] 1. Remote sensing datasets for landfills are still relatively scarce. They are usually independently generated by research projects or cooperative institutions, and data acquisition channels are mostly specific collaborations or research applications, which makes them less open.

[0008] 2. Existing identification methods are limited to specific regions and rely on external features surrounding the landfill rather than focusing on the landfill's own characteristics. The results only provide a probabilistic map of the landfill's distribution, limiting the accuracy and applicability of the identification results.

[0009] 3. Existing identification methods rely too much on manual experience and judgment in landfill feature extraction, which affects the accuracy and efficiency of identification. Summary of the Invention

[0010] The technical problem to be solved by the present invention is to address the shortcomings of the existing technology and provide an automatic recognition method for solid waste landfill remote sensing images based on deep learning, thereby improving the accuracy of solid waste landfill identification.

[0011] The present invention provides a method for automatically identifying remote sensing images of solid waste landfills based on deep learning, the method comprising the following steps:

[0012] Step 1: Collect remote sensing image data of solid waste landfill;

[0013] Step 2: Integrate the solid waste landfill remote sensing image data into the dataset RSD46-WHU to construct a new dataset RSSCD47-GU;

[0014] Step 3: Build a GSFF-VIT model based on the improved VIT model, train and test the RSSCD47-GU dataset using the GSFF-VIT model and multiple deep learning models to obtain a confusion matrix, and calculate the recognition index of each model based on the confusion matrix;

[0015] Step 4: Compare and analyze the models based on the confusion matrix and recognition index to determine the effectiveness of the GSFF-VIT model.

[0016] Preferably, the steps of constructing the GSFF-VIT model are:

[0017] Step 1: Use the image in the dataset RSSCD47-GU as the input image x , the input image x Divide into P small blocks, the size of each small block is , and then map it to the D-dimensional feature space to obtain the block feature image ; Among them, H and W are the input images x height and width;

[0018] Step 2: Add a category token to the block feature image The starting position of the new block feature image is obtained ;

[0019] Step 3: Feature image for the new block Add position embeddings to each small block in , as well as the category token;

[0020] Step 4: For the new block feature image The positional embeddings of the image tokens are upsampled to match the new feature shape;

[0021] Step 5: The new block feature image after sampling The position of the image token is embedded with the new block feature image The category tokens are concatenated to obtain a tensor;

[0022] Step 6: Construct a bottle neck structure to fuse the information of the tensor;

[0023] Step 7: Add the information fused tensor to the new block feature image , obtain the block feature image ;

[0024] Step 8: Feature image of the block Perform the exit operation to obtain the image features ;

[0025] Step 9: Process image features through L-layer Transformer blocks , get the output of L Transformer blocks;

[0026] Step 10: Perform LayerNorm normalization on the output of the Transformer block of the last layer to obtain the normalized image features x final ;

[0027] Step 11: Extract the normalized image features x final Characteristics of the category token x class , according to the characteristics x class For the input image x to classify.

[0028] Preferably, the information of the tensor is fused, specifically:

[0029] The dimension of the tensor is transformed, and the features of the transformed tensor are fused and the dimension is amplified using two linear layers, and then the dimension is transformed back to the original dimension.

[0030] Preferably, the recognition indicators include Top-1 Accuracy, Top-5 Accuracy, Mean Precision and Mean Recall; wherein,

[0031] Top-1 Accuracy indicates the proportion of samples whose predicted category with the highest probability is consistent with the true category in the total number of samples;

[0032] Top-5 Accuracy indicates the proportion of samples containing the true category in the top five categories predicted by the model with the highest probability to the total number of samples;

[0033] Mean Precision represents the average precision;

[0034] Mean Recall represents the average recall rate.

[0035] Preferably, the Top-1 Accuracy is calculated by the following formula:

[0036] ;

[0037] Among them, TP i Indicates that the model correctly predicts the sample of true category i as category i; TN i Indicates that the model correctly predicts samples whose true category is not i as non-i category; FP i Indicates that the model mistakenly predicts a sample whose true category is not i as category i; FN iIt means that the model mistakenly predicts the sample of the true category i as a non-i category; the category is the collection scene of solid waste landfill remote sensing image data.

[0038] Preferably, the Mean Precision is calculated by the following formula:

[0039] ;

[0040] in, k is the total number of categories.

[0041] Preferably, the Mean Recall is calculated by the following formula:

[0042] .

[0043] Preferably, the multiple deep learning models include convolutional neural network models and Transformer architecture models, specifically: MobileNetV2, ResNet18, Vgg16, ResNet34, ResNet50, Swin Transformer, VIT.

[0044] Preferably, step four is specifically as follows: visualizing the recognition indicators of all models in graphs, analyzing the visualized graphs to determine whether the GSFF-VIT model is superior to other models in processing solid waste landfill remote sensing image data; by analyzing the confusion matrix, finding the category whose error exceeds the set threshold in each model as the target category, extracting the confidence of the target category from the prediction results of each model for the data set RSSCD47-GU, and judging the effectiveness of the GSFF-VIT model through visual analysis of the confidence.

[0045] Preferably, a model with the best confidence is selected from the convolutional neural network model as a comparison model, and multiple categories other than the target category are selected from all categories as comparison categories. The confidence of the comparison model and the GSFF-VIT model based on the comparison categories is visualized and analyzed to further judge the effectiveness of the GSFF-VIT model.

[0046] Beneficial effects

[0047] The advantages of the present invention are:

[0048] 1. The RSD46-WHU dataset was improved, and a multi-regional solid waste landfill remote sensing image dataset was created, filling the gap in existing landfill remote sensing data.

[0049] 2. It has achieved a transition from manual feature extraction to data-driven end-to-end feature extraction, solving the problems of difficult and inefficient feature extraction in the landfill identification process. At the same time, the model can be quickly applied to landfill identification in other regions, improving the efficiency of identification and application.

[0050] 3. A deep learning model for automatic identification of large-scale solid waste landfills was established. The Transformer architecture was introduced into the landfill identification task, and the Vision Transformer model was optimized to enable it to more effectively focus on the landfill's own characteristics, thereby improving the accuracy of solid waste landfill identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of the automatic recognition method of remote sensing images of solid waste landfills according to the present invention;

[0052] Figure 2 (a)-(b) shows the solid waste landfill remote sensing image data collection process and image display diagrams of the present invention;

[0053] Figure 3 This is a schematic diagram of an example of the RSSCD47-GU dataset of the present invention;

[0054] Figure 4 Schematic diagram of the GSFF-VIT model of the present invention;

[0055] FIG5( b ) is a schematic diagram of the classification confusion matrix of the ResNet18 model of the present invention on the RSSCD47-GU dataset;

[0056] Figure 5 (c) is a schematic diagram of the classification confusion matrix of the ResNet34 model of the present invention on the RSSCD47-GU dataset;

[0057] FIG5( d ) is a schematic diagram of the classification confusion matrix of the VIT model of the present invention on the RSSCD47-GU dataset;

[0058] Figure 5 (e) is a schematic diagram of the classification confusion matrix of the MobileNetV2 model of the present invention on the RSSCD47-GU dataset;

[0059] FIG5( f ) is a schematic diagram of the classification confusion matrix of the Swin-Transformer model of the present invention on the RSSCD47-GU dataset;

[0060] Figure 5(g) is a schematic diagram of the classification confusion matrix of the ResNet50 model of the present invention on the RSSCD47-GU dataset;

[0061] FIG5(h) is a schematic diagram of the classification confusion matrix of the Vgg16 model of the present invention on the RSSCD47-GU dataset;

[0062] FIG5 (i) is a schematic diagram of the classification confusion matrix of the GSFF-VIT model of the present invention on the RSSCD47-GU dataset;

[0063] Figure 6 This is a visualization diagram showing the results of different models of the present invention for categories (0) to (4) in Table 2;

[0064] Figure 7 This is a visualization diagram showing the results of different models of the present invention for categories (5) to (9) in Table 2;

[0065] Figure 8 This is a visualization diagram showing the effectiveness analysis of the GSFF-VIT model of the present invention. DETAILED DESCRIPTION

[0066] The present invention will be further described below in conjunction with the embodiments, but this does not constitute any limitation to the present invention. Any limited number of modifications made by anyone within the scope of the claims of the present invention are still within the scope of the claims of the present invention.

[0067] The present invention provides an automatic identification method for remote sensing images of solid waste landfills based on deep learning. The method mainly has six key core steps, and its process is as follows: Figure 1 The specific steps are as follows:

[0068] Step 1: Determine the research scope.

[0069] This step aims to collect remote sensing images of landfills from various regions to expand the dataset's feature diversity and provide rich feature information for subsequent model learning. The data collection process follows a scientific and systematic approach. Regarding data distribution, to ensure representativeness and improve the model's generalization capabilities, special consideration was given to the specificity, representativeness, and data availability of different regions, striving to ensure that the selected landfills cover all regions of the country. During the selection process, a special focus was placed on provinces with significant differences in topography, economic development, and solid waste management practices, ensuring that the dataset includes landfills with diverse characteristics.

[0070] The detailed data collection steps are shown in Figure 2(a). During the data collection process, we searched for target site information using keywords such as "landfill," "disposal site," and "abandoned soil site" through official public channels, news reports, Baidu Maps, and AutoNavi Maps to ensure data authenticity and validity. This information provided key information for the study, including the project name, location, scale, and operating status of each target site, laying the foundation for subsequent site geographic coordinate query and image data collection. After the site information was generally confirmed, the AutoNavi Map coordinate picker was used to query the latitude and longitude coordinates of each target site. Taking into account the fact that some site information was limited or invalid, making it impossible to obtain accurate latitude and longitude coordinates, we ultimately obtained accurate latitude and longitude coordinates for a total of 70 solid waste landfills from 23 provinces in China.

[0071] Step 2: Collect remote sensing image data of solid waste landfill.

[0072] Based on this coordinate information, Google Earth was used to sample each landfill. Figure 2(b) shows a sample remote sensing image of a landfill, with the main facilities of the landfill indicated. A solid waste landfill consists of several disposal units, structures, and areas, primarily including waste pretreatment facilities, a waste landfill area, a garbage dam, leachate collection and treatment facilities, landfill gas drainage facilities, and auxiliary engineering. The landfill area is the core area of ​​the landfill and typically appears as an irregular, uneven pile of waste with a darker surface. The top of the pile is covered with soil or synthetic film, and the covered area is smooth with no significant protrusions. The soil cover is typically light brown or yellowish-brown, while the synthetic film cover is dark black or dark gray. Garbage dams are typically located around the perimeter of the landfill to stabilize the pile structure. The road system is well-defined, often in a circular or crisscross pattern, making it convenient for large transport vehicles to transport waste or materials. Large landfills usually have leachate and gas treatment plants at the edges, and office, monitoring, and management buildings in the entrance area. These buildings appear as small and regular geometric shapes in remote sensing images.

[0073] To fully utilize the data, we conducted multiple time series sampling of the same site, which showed significant changes over time, as different samples. Ultimately, we collected a total of 586 landfill remote sensing images.

[0074] Step 3: Dataset construction.

[0075] This step combines the remote sensing image data collected in step 1 and integrates it into the RSD46-WHU dataset, completing the construction of the solid waste landfill category dataset.

[0076] In the present invention, the model learns data features from the image data by itself. The quality of the dataset directly affects the accuracy of subsequent model training and the final image recognition results. Therefore, the construction of the dataset is a critical task. The dataset of the present invention has a wide coverage, covering landfill image data from many regions across the country. RSD46-WHU is a large-scale open dataset, mainly used for remote sensing image scene classification. The dataset is collected from multiple sources such as Google Earth and Tianditu. The ground resolution of most categories in the dataset reaches a high accuracy of 0.5 meters, and other categories are about 2 meters. Each category contains 500 to 3000 images, totaling 117,000 images. The dataset covers 46 different categories, providing a rich data resource for remote sensing image analysis.

[0077] According to the construction mode of RSD46-WHU dataset, 586 landfill images were screened and constructed into a new dataset Remote Sensing Scene Classification Dataset GuangzhouUniversity 47 (RSSCD47-GU) in an 8:2 ratio. The images were randomly divided into training set and test set: training set 467; test set 119. The dataset sample is as follows: Figure 3 As shown, there are 47 categories in total. Figure 3In the figure, the serial numbers and their photos represent: (01) Airplane, (02) Airport, (03) Artificial dense forest land, (04) Artificial sparse forest land, (05) Bare land, (06) Basketball Court, (07) Blue Structured factory Building, (08) Building, (09) Construction Site, (10) Cross River Bridge, (11) Crossroads, (12) Dense Tall Building, (13) Dock, (14) Fish Pond, (15) Footbridge, (16) Graff, (17) Grassland, (18) Low Scattered Building, (19) Regular Farmland, (20) Medium Density Scattered Building, Building (medium-density dispersed building), (21) Medium-density structured building, (22) Natural dense forest land, (23) Natural sparse forest land, (24) 0il tank, (25) 0verpass, (26) Parking lot, (27) Plastic Greenhouse, (28) Playground, (29) Railway, (30) Red structured factory building, (31) Refinery, (32) Regular Farmland, (33) Scattered blue roof factory building, (34) Scattered red roof factory building, (35) Sewage Plant type one, (36) Sewage Plant type two, (37) Ship, (38) Solar Powerstation solar power station, (39) sparse residential area, (40) square, (41) steel smelter, (42) storage land, (43) tennis court, (44) thermal power plant, (45) vegetable plot, (46) waste landfill, (47) water.

[0078] Step 4: Training and development of deep learning models for identifying solid waste landfills.

[0079] This step aims to train and test based on classic deep learning models. The present invention uses 7 classic deep learning models and the GSFF-VIT model built based on VIT as the basis of the landfill automatic identification method, and applies them to the data set RSSCD47-GU for training and testing, thereby improving the recognition accuracy while realizing the automatic identification of solid waste landfills. Among them, the 7 classic deep learning models include two major categories, namely convolutional neural network and Transformer architecture models, specifically MobileNetV2, ResNet18, ResNet34, ResNet50, Vgg16, Swin-Transformer, and VIT.

[0080] These models were trained and tested on a platform with the following hardware configuration: an Intel i7-12700H CPU, 32GB of RAM, and an RTX 2080Ti GPU. All code was written and executed in the PyTorch environment. During training, the batch size of the model was set to 16 during the freezing phase and 8 during the thawing phase. The initial learning rate was 1e-3 (when using the Adam optimizer). The model took 256×256 images as input and was trained for 200 epochs. The Adam optimizer was used in conjunction with a learning rate scheduler. During training, cosine annealing was used as the learning rate decay strategy.

[0081] The MobileNetV2, ResNet18, Vgg16-bn, ResNet34, ResNet50, Swin-Transformer, VIT, and GSFF-VIT models were trained and tested on the RSSCD47-GU dataset, and the confusion matrix and confidence score were finally obtained. The confusion matrix is ​​shown in Figure 5 (b) to (i). In Figure 5 (b) to (i), (01) to (47) correspond to the categories (01) to (47) in Figure 2. The recognition indicators of each model (including Top-1 Accuracy, Top-5 Accuracy, Mean Recall, and Mean Precision) are calculated based on the results of the confusion matrix.

[0082] In the multi-category remote sensing image classification task, the output of the model can be divided into four cases similar to the binary classification problem, namely, true positive (TP), true negative (TN), false positive (FP) and false negative (FN). For each category i, TP i Indicates that the model correctly predicts the sample of true category i as category i; TN i Indicates that the model correctly predicts samples whose true category is not i as non-i category; FP i Indicates that the model mistakenly predicts a sample whose true category is not i as category i; FN i It means that the model mistakenly predicts samples whose true category is i as non-i category.

[0083] The comparison of model performance is based on several key indicators. The effectiveness of different models in automatic identification of solid waste landfills is evaluated through quantitative analysis. The main indicators include the following:

[0084] Top-1 Accuracy: This indicates the proportion of samples whose most likely category (i.e., the category with the highest probability) predicted by the model matches the true category. This metric can well illustrate the accuracy of the model on the entire dataset. The calculation formula is shown in Formula 1:

[0085] (Formula 1).

[0086] Top-5 Accuracy: This metric indicates the proportion of samples whose true class is included in the top five most likely classes predicted by the model. This metric functions similarly to Top-1 Accuracy, except that even if the most likely prediction is not the true class of the sample, the prediction is considered correct as long as the true class is included in the top five most likely predictions.

[0087] Mean Precision: Average precision. Precision represents the ratio of the number of samples whose true category is i (TP) to the number of samples predicted to be i by the model. Mean Precision is the average of the Precisions of all categories. The calculation formula is shown in Formula 2:

[0088] (Formula 2).

[0089] Mean Recall: Average recall rate. Recall represents the proportion of samples with actual category i that can be correctly predicted by the model as category i. Mean Recall is the average of the recalls of all categories. The calculation formula is shown in Formula 3:

[0090] (Formula 3)

[0091] In the present invention, the GSFF-VIT model design and construction method are as follows.

[0092] For the Remote Sensing Scene Classification (RSSC) task, due to the variability of remote sensing target scales, using VIT’s patch embedding method to split remote sensing images may lose important information. Therefore, this paper proposes to use the GSFF-VIT model to enhance VIT’s fusion of semantic feature information and spatial coordinates contained in image patches. The schematic diagram of the GSFF-VIT model is shown in the figure. Figure 4 As shown, the specific construction steps are as follows:

[0093] For the input image x , of shape [B,C,H,W], where B is the batch size, C is the number of channels, and H and W are the height and width of the image, respectively.

[0094] Step 1: Patch Embedding.

[0095] The input image x Divide into P patches, the size of each patch is , and then mapped to the D-dimensional feature space. The specific formula is as follows: ,in The dimensions are [B, N, D], is the total number of patches, and D is the feature dimension.

[0096] Step 2: Add Class Token.

[0097] Add a class token to At the beginning of the class token, the size is [1,1,D]. The specific formula is as follows:

[0098] ,in The size of is [B,N+1,D].

[0099] Step 3: Positional Embedding.

[0100] for Each patch and class token in is embedded as follows:

[0101] ; .

[0102] Step 4: Interpolate Positional Embedding.

[0103] right The position embedding of the image token in is upsampled to match the new feature shape. It is shown in the following formula:

[0104] .

[0105] Step 5: Concatenate Positional Embedding.

[0106] After sampling The position of the image token is embedded with The category tokens are concatenated to obtain a tensor. The specific formula is as follows:

[0107] Among them, pos_embed is a tensor that combines category (semantic) information and position information.

[0108] Step 6: Build a bottle neck structure to fuse the tensor information. First, transform the tensor dimensions: y = y.view(B,C,1). Then, use two linear layers to fuse the features and increase the dimensions. Finally, transform the dimensions back to the original dimensions to complete the fusion.

[0109] Step 7: Residual Learning.

[0110] Add the information fused tensor to the new block feature image The specific formula is as follows: .

[0111] Step 8: Dropout.

[0112] Block feature image Perform the dropout operation, as shown in the following formula: .

[0113] Step 9: Transformer Blocks processing.

[0114] pass Transformer Blocks processing of the layer For a single Transformer block, the specific formula is as follows:

[0115] ,in is the output of the i-th layer Transformer block, .

[0116] Step 10: LayerNorm.

[0117] Perform LayerNorm normalization on the output of the last layer of Transformer Blocks. The specific formula is as follows:

[0118] ,in x final is the normalized feature.

[0119] Step 11: Extract Class Token.

[0120] extract x final The features of the class token are used for classification. The specific formula is as follows: .

[0121] Finally, the GSFF-VIT model outputs classtoken features, processed and normalized by Transformer Blocks, for subsequent classification tasks. The above 11 steps complete the improvement of the VIT model and construct the GSFF-VIT model.

[0122] Step 5: Evaluation and comparison of model recognition performance.

[0123] This step aims to evaluate the accuracy of the model test results in the previous step to clarify the recognition capabilities of each model. Specifically, the recognition indicators are visualized, as shown in Table 1.

[0124]

[0125] The ResNet18, ResNet34, ResNet50, MobileNetV2, Vgg16, Swin Transformer, VIT, and GSFF-VIT models were trained and tested on the RSSCD47-GU dataset. Their confusion matrices are shown in Figure 5. The recognition metrics (including Top-1 Accuracy, Top-5 Accuracy, Mean Recall, and Mean Precision) for each model were calculated based on the confusion matrix results, and the results are shown in Table 1.

[0126] The metric analysis in Table 1 demonstrates that the GSFF-VIT model demonstrates significant advantages in image recognition tasks. Its Top-1 Accuracy and Top-5 Accuracy are 93.51% and 99.78%, respectively, the highest among all models in the table, demonstrating exceptionally high recognition accuracy and confidence. Furthermore, GSFF-VIT's Average Recall and Average Precision reach 93.53% and 93.52%, respectively, also outperforming other models. This demonstrates its superior performance in identifying positive examples and predicting positive examples accurately. The combined performance of these metrics demonstrates the strong competitiveness and application potential of GSFF-VIT in the field of image recognition.

[0127] Step 6: Visual analysis of the performance of the solid waste landfill automatic identification model.

[0128] The first step is to visually analyze the easily confused categories of each model.

[0129] Summarizing the confusion matrix results, we obtain the accuracy results shown in Table 2. As can be seen from Figure 5 and Table 2, there are 10 categories with error rates exceeding 30%. This paper extracts the top 10 categories of targets with the highest errors for visual analysis to evaluate the performance of different models.

[0130]

[0131] It should be noted that the numbers before each category in Table 2 above are table numbers, which are not equivalent to Figure 3 The serial numbers corresponding to each category.

[0132] Visual analysis results are as follows Figure 6 and Figure 7For better analysis, we correspond the top ten categories with the highest errors (Medium density scattered building, Construction site, Artificial dense forestland, Building, Artificial sparse forest land, Thermal power plant, Sparseresidential area, Refinery, Bare land, Medium density structured building) to (0) to (9) in Table 2 above, and encode the model. Since the prediction results include the prediction category and confidence, this part of the information is summarized in Table 3. Specific details are shown in Table 3.

[0133]

[0134] according to Figure 6 From the analysis, combined with Table 3, we can see that for the (0) Medium density scattered building category, the classification results of each model except GSFF-VIT are all wrong. The proposed GSFF-VIT not only captures the image information of the building part, but also achieves correct classification. Figure 6 In the (1) Construction site category, all models are classified correctly and achieve high classification accuracy (arrows); Figure 6 In the (2) Artificial dense forest land category, GSFF-VIT not only correctly classified the classification, but also had the highest confidence. Figure 6 In the (3) Building category, the distribution of key targets is captured, but the classification errors are missing; Figure 6 In the (4) Artificial sparse forest land category, GSFF-VIT also performs superiorly.

[0135] according to Figure 7From the analysis and Table 3, we can see that for the (5) Thermal power plant category, all models classified correctly, and the GSFF-VIT confidence score was also one of the highest. For the (6) Sparse residential area category, the proposed GSFF-VIT performed best and had the highest confidence score. For the (7) Refinery category, the GSFF-VIT performance was only better than ResNet18. For the (8) and (9) categories, the GSFF-VIT classification performance for Bare land and Medium density structured building was also the highest.

[0136] comprehensive Figure 6 、 Figure 7 From Table 3, we can see that the proposed GSFF-VIT has the best overall performance and performs best in almost all high error categories.

[0137] Step 2: Visual analysis of GSFF-VIT on landfill categories.

[0138] The first step verifies the effectiveness of the proposed model across all datasets. To further demonstrate the superiority of the proposed GSFF-VIT, this step organizes comparative ablation experiments to discuss the effectiveness of the proposed components. Since the classification target of this invention is primarily wastelandfill, relevant landfill data is randomly selected, corresponding to (10) to (14) in Table 2. VIT and GSFF-VIT are used for classification and recognition, and the difference in the performance of the two models is discussed.

[0139] To further analyze the effectiveness of the proposed GSFF-VIT on landfill types, the present invention further demonstrates the performance of GSFF-VIT and VIT on different landfill images, as shown in the following example. Figure 8 As shown, we summarize the corresponding classification accuracy in Table 4. From Table 4, we can see that GSFF-VIT is slightly better than the original VIT in terms of confidence. Therefore, the present invention uses visual analysis to explain the effectiveness of the proposed GSFF-VIT, as shown in Figure 8 In column (10), VIT focuses on the surrounding farmland (arrow A), while GSFF-VIT focuses more on the landfill area in the middle and has a higher confidence level; Figure 8 In column (11), VIT also focuses on the surrounding wasteland, but GSFF-VIT captures the boundary information of the key buildings and landfills (arrows BC); Figure 8 In column (12), both VIT and GSFF-VIT focus on the key area of ​​the landfill (EF area), but VIT allocates attention weight to the land boundary on the other side of the river (arrow D); Figure 8In column (13), VIT focuses on the surrounding farmland (arrow H), but GSFF-VIT focuses on the key target - the landfill area (arrow G); Figure 8 In column (14), VIT still focuses on the surrounding wasteland, while GSFF-VIT pays attention to the regional information of the landfill (arrow IJ).

[0140]

[0141] Therefore, in the landfill identification task, compared with the traditional VIT model, the GSFF-VIT model performs better in the ability to focus on regions and facilities and the accuracy of feature capture, thereby improving the classification accuracy and practical application value.

[0142] The above is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the structure of the present invention. These modifications and improvements will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A method for automatic identification of remote sensing images of solid waste landfills based on deep learning, characterized in that: The method comprises the following steps: Step 1: Collect remote sensing image data of solid waste landfill; Step 2: Fuse the solid waste landfill remote sensing image data into the dataset RSD46-WHU to construct a new dataset RSSCD47-GU; Step 3: Build a GSFF-VIT model based on the improved VIT model, train and test the RSSCD47-GU dataset using the GSFF-VIT model and multiple deep learning models to obtain a confusion matrix, and calculate the recognition index of each model based on the confusion matrix; Step 4: Compare and analyze the models based on the confusion matrix and recognition index to determine the effectiveness of the GSFF-VIT model; The steps for constructing the GSFF-VIT model are: Step 1: Use the image in the dataset RSSCD47-GU as the input image x , the input image x Divide into P small blocks, the size of each small block is , and then map it to the D-dimensional feature space to obtain the block feature image ; Among them, H and W are the input images x height and width; Step 2: Add a category token to the block feature image The starting position of the new block feature image is obtained ; Step 3: Feature image for the new block Add position embeddings to each small block in , as well as the category token; Step 4: For the new block feature image The positional embeddings of the image tokens are upsampled to match the new feature shape; Step 5: The new block feature image after sampling The position of the image token is embedded with the new block feature image The category tokens are concatenated to obtain a tensor; Step 6: Construct a bottle neck structure to fuse the information of the tensor; Step 7: Add the information fused tensor to the new block feature image , obtain the block feature image ; Step 8: Feature image of the block Perform the exit operation to obtain the image features ; Step 9: Process image features through L-layer Transformer blocks , get the output of L Transformer blocks; Step 10: Perform LayerNorm normalization on the output of the Transformer block of the last layer to obtain the normalized image features x final ; Step 11: Extract the normalized image features x final Characteristics of the category token x class , according to the characteristics x class For the input image x to classify.

2. The method for automatic identification of remote sensing images of solid waste landfills based on deep learning according to claim 1 is characterized in that: The information of the tensor is fused, specifically: The dimension of the tensor is transformed, and the features of the transformed tensor are fused and the dimension is amplified using two linear layers, and then the dimension is transformed back to the original dimension.

3. The method for automatic identification of remote sensing images of solid waste landfills based on deep learning according to claim 1, characterized in that: The recognition indicators include Top-1 Accuracy, Top-5 Accuracy, Mean Precision and Mean Recall; Top-1 Accuracy indicates the proportion of samples whose predicted category with the highest probability is consistent with the true category in the total number of samples; Top-5 Accuracy indicates the proportion of samples containing the true category in the top five categories predicted by the model with the highest probability to the total number of samples; Mean Precision represents the average precision; Mean Recall represents the average recall rate.

4. The method for automatic identification of remote sensing images of solid waste landfills based on deep learning according to claim 3 is characterized in that: The Top-1 Accuracy is calculated by the following formula: ； Among them, TP i Indicates that the model correctly predicts the sample of true category i as category i; TN i Indicates that the model correctly predicts samples whose true category is not i as non-i category; FP i Indicates that the model mistakenly predicts a sample whose true category is not i as category i; FN i It means that the model mistakenly predicts the sample of the true category i as a non-i category; the category is the collection scene of solid waste landfill remote sensing image data.

5. The method for automatic identification of remote sensing images of solid waste landfills based on deep learning according to claim 4 is characterized in that: The Mean Precision is calculated by the following formula: ； in, k is the total number of categories.

6. The method for automatic identification of remote sensing images of solid waste landfills based on deep learning according to claim 5, characterized in that: The Mean Recall is calculated by the following formula: 。 7. The method for automatic identification of remote sensing images of solid waste landfills based on deep learning according to claim 1, characterized in that: The multiple deep learning models include convolutional neural network models and Transformer architecture models, specifically: MobileNetV2, ResNet18, Vgg16, ResNet34, ResNet50, Swin Transformer, VIT.

8. The method for automatic identification of remote sensing images of solid waste landfills based on deep learning according to claim 1, characterized in that: Step four is as follows: visualize the recognition indicators of all models in graphs, analyze the visualized graphs to determine whether the GSFF-VIT model is superior to other models in processing solid waste landfill remote sensing image data; by analyzing the confusion matrix, find the category in each model whose error exceeds the set threshold as the target category, extract the confidence of the target category from the prediction results of each model for the data set RSSCD47-GU, and determine the effectiveness of the GSFF-VIT model through visual analysis of the confidence.

9. The method for automatic identification of remote sensing images of solid waste landfills based on deep learning according to claim 8, characterized in that: A model with the best confidence is selected from the deep learning model as a comparison model, and multiple categories other than the target category are selected from all categories as comparison categories. The confidence of the comparison model and the GSFF-VIT model based on the comparison categories is visualized and analyzed to further determine the effectiveness of the GSFF-VIT model.

Images