A method for detecting an informal garbage dumping site and a computer-readable storage medium

Abstract

The present invention relates to a detection method for informal garbage dumping sites and a computer-readable storage medium, wherein the detection method specifically comprises: constructing three-dimensional point cloud data and orthophoto images of a city block to be tested by aerial photography; classifying and labeling informal garbage dumping sites in the three-dimensional point cloud data and orthophoto images, and generating garbage image detection sample sets and garbage point cloud detection sample sets; generating garbage classification functional areas by calculating the feature map of points of interest of each block and extracting the feature map of natural landscapes; establishing image detection models and point cloud detection models for informal garbage dumping sites respectively; extracting suspicious point map sequence S1 of informal garbage dumping sites by the image detection model, and further detecting the suspicious point map sequence S1 by using the point cloud detection model, and outputting the recognition result by weighted fusion. The method realizes high-precision extraction of informal garbage dumping sites, and the extraction result has high accuracy and fast processing speed, which meets the high efficiency and real-time performance of practical applications.

Description

Technical Field

[0001] The present invention relates to the fields of unmanned aerial vehicle remote sensing and artificial intelligence deep learning, and in particular to a method for detecting informal garbage dumping sites based on a generalized receptive field and a computer-readable storage medium. Background Art

[0002] Traditional monitoring methods for informal garbage dumping sites usually rely on manual field surveys, which require a lot of manpower and material resources, have low supervision efficiency and are difficult to adapt to the rapid changes in the urban landscape. With the rapid development of remote sensing technology and artificial intelligence, new opportunities have emerged in the field of garbage detection. Remote sensing technology, especially drones and satellite technology, can provide high-resolution remote sensing image data and three-dimensional point cloud data, comprehensively observe the distribution of urban garbage dumping sites from multiple perspectives, and provide new ideas for monitoring and classification of informal garbage dumping sites. For example, the Chinese patent application with application number CN202211219087.2 discloses a method for identifying and determining illegal garbage based on drones and artificial intelligence. It collects high-definition videos through drones, then performs inter-frame difference processing on the high-definition videos collected by drones to obtain effective high-definition pictures, and then transmits the obtained high-definition pictures to the ground workstation through 5G technology, further pre-processes the pictures, and finally calculates the pre-processed pictures through artificial intelligence algorithms, thereby realizing the identification and determination of garbage. However, the variety and complexity of garbage dumping sites, as well as their differentiated distribution in different urban functional areas, make the detection of informal garbage dumping sites face technical difficulties and challenges. How to combine regional characteristics to accurately identify the location and type of garbage spots in large-scale remote sensing images has become an urgent problem to be solved. Summary of the invention

[0003] The primary technical problem to be solved by the present invention is to provide a method for detecting informal garbage dumping sites in the above-mentioned prior art, which can identify the precise garbage spot locations and garbage types in remote sensing images in combination with regional features.

[0004] A further technical problem to be solved by the present invention is to provide a computer-readable storage medium storing the above-mentioned informal garbage dumping site detection method.

[0005] The technical solution adopted by the present invention to solve the above-mentioned primary technical problem is: a method for detecting informal garbage dumping points, which is used to detect informal garbage dumping points in a city block to be detected, and is characterized by comprising the following steps:

[0006] Step 1: construct three-dimensional point cloud data and orthophotos of the city blocks to be tested based on aerial images, and obtain road network data and point of interest data of the city blocks to be tested;

[0007] Step 2: Classify and annotate the three-dimensional point cloud data of the urban blocks to be tested and the informal garbage dumping points in the orthophotos by means of visual interpretation and manual interpretation, and generate a garbage image detection sample set and a garbage point cloud detection sample set;

[0008] Step 3: Segment and extract the basic blocks of the city to be tested based on the road network data of the city blocks to be tested, and calculate the feature map of the points of interest of each basic block; extract the natural landscape feature map by classifying the objects through remote sensing interpretation of the orthophoto; then use the feature map of the points of interest and the natural landscape feature map to perform cluster analysis on the basic blocks of the city to be tested, and obtain the probability feature map of the garbage classification functional area of ​​the city blocks to be tested. , and generate garbage classification functional areas of basic blocks in the city to be tested;

[0009] Step 4: Use the garbage image detection sample set generated in step 2 to train the target detection model, extract pixel features in the garbage image, use the garbage classification functional area probability feature map P generated in step 3 to extract scene features, and then fuse pixel features and scene features to construct an image detection model for informal garbage dumping points in the basic blocks of the city to be tested;

[0010] Step 5: construct a point cloud detection model for the basic blocks of the city to be tested, and use the garbage point cloud detection sample set generated in step 2 to train the point cloud detection model;

[0011] Step 6: Extract the suspicious spot sequence S1 of the informal garbage dumping point according to the informal garbage dumping point image detection model in step 4, and then use the point cloud detection model constructed in step 5 to further detect the suspicious spot sequence S1, and output the recognition result through weighted fusion.

[0012] As an improvement, in step 1, drone oblique photography technology is used to obtain high-resolution aerial images of the city blocks to be tested, and then real-life 3D modeling software is used to process the aerial images to construct 3D point cloud data and orthophotos of the city blocks to be tested; road network data of the city blocks to be tested are obtained based on the open street map, and points of interest of the city blocks to be tested are obtained based on Baidu map POI.

[0013] Further improved, the step 2 comprises:

[0014] Step 2-1, use the geographic information application software arcgis to mark the circumscribed rectangle of the informal garbage dumping site on the orthophoto of the city block to be tested, and assign a garbage category label to each informal garbage dumping site based on the visual features, contextual information and possible garbage sources of the informal garbage dumping site. The garbage category labels include five categories: domestic garbage, construction garbage, industrial garbage, ecological garbage, and mixed garbage; crop the orthophoto of the city block to be tested to a size of 512×512 using a sliding window method, and use a .json file to store the location and category information of the informal garbage dumping site;

[0015] Step 2-2: Use the SementicKITTI point cloud annotation tool to mark the point cloud set of each informal garbage dumping point in the three-dimensional point cloud data of the urban block to be tested, and assign corresponding garbage category labels.

[0016] Further improved, the step 3 comprises:

[0017] Step 3-1, in the road network data of the city block to be tested, expressways, trunk roads, main roads, secondary roads, third-level roads with a length greater than 150m and unclassified roads are selected as the basic road network data of the city block to be tested;

[0018] Step 3-2: In the basic road network data of the city block to be tested obtained in step 3-1, the two-lane roads are merged into single-lane roads, and the road spaces including expressways, trunk roads, main roads, secondary roads, and branch roads are removed from the basic road network data of the city block to be tested to obtain independently distributed plot units, and then the merged area is less than 30,000m 2 The basic block distribution map of the city to be tested is obtained by

[0019] Step 3-3: According to the correlation between the category names of the points of interest in the city blocks to be tested and the garbage categories, the points of interest in the city blocks to be tested are reclassified into five categories: residential, commercial, industrial, ecological and transportation, and the feature map of the points of interest in each basic block is calculated. :

[0020]

[0021] in, is the serial number of the point of interest, is the block number, For the The first The number of interest points of the class, For the The number of points of interest of all types in the block, For the The first The sum of the areas occupied by the points of interest of the same type, For the The area of ​​the block;

[0022] Step 3-4: Use the land feature classification method of remote sensing interpretation to classify the basic blocks of the city to be tested. The land feature classification categories include farmland, forest and grassland, buildings, roads, bare land, water areas and others, a total of seven categories, and extract the natural landscape feature map of the basic blocks of the city to be tested. :

[0023]

[0024] in is the classification number of the land feature, For the The first The number of patches classified as land features, For the The total number of patches in the block, For the The first The sum of the areas occupied by the elements of this class, For the Mean area of ​​the feature classification; For the The first The sum of the perimeters of the feature classifications, For the Mean perimeter of the feature classification;

[0025] Step 3-5: Use the feature map of interest points obtained in step 3-3 And the natural landscape feature map obtained in steps 3-4 , a Gaussian mixture model is used to perform cluster analysis on the basic block distribution map of the tested city, and the probability characteristic map of garbage classification functional areas of the tested city blocks is obtained. ;

[0026] Then calculate the probability characteristic map of garbage classification functional areas The extreme value of the garbage classification functional area category of the basic block of the city to be tested is obtained , the categories of garbage classification functional areas include residential areas, commercial areas, industrial areas, ecological areas, mixed functional areas, and other areas:

[0027]

[0028] in, It is the category number of the garbage classification functional area. The value of can be any one of residential area, commercial area, industrial area, ecological area, mixed function area, and other areas. Indicates Blocks belong to category The probability of Indicates calculating extreme values ​​in the probability of all garbage classification functional area categories.

[0029] Further improved, the step 4 specifically includes:

[0030] Step 4-1: Use the Yolov8 target detection model to take the junk image detection sample set generated in step 2 as input data, extract pixel features from the junk image, and represent the original image in the junk image detection sample set as , corresponding to the pixel features of the garbage image It is expressed as:

[0031]

[0032] in, Contains features of 3 sizes, and the features of each size are expressed as , m=1,2,3;

[0033] Step 4-2, construct a scene feature enhancement module EPB, the scene feature enhancement module EPB includes three parallel branches - a first branch, a second branch and a third branch, wherein the first branch includes a first 1×1 convolution module, a first RFB module and a second 1×1 convolution module connected in sequence, the second branch includes a third 1×1 convolution module, a second RFB module and a fourth 1×1 convolution module connected in sequence, the third branch includes a fifth 1×1 convolution module, a third RFB module and a sixth 1×1 convolution module connected in sequence, the first RFB module, the second RFB module and the third RFB module The modules have the same structure, and all include a Fourier transform module, a 3×3 channel-by-channel convolution module, an addition module, and an inverse Fourier transform module, wherein the input end of the Fourier transform module is the input end of the entire module, the output end of the Fourier transform module is connected to the first input end of the addition module, the output end of the Fourier transform module is also connected to the input end of the 3×3 channel-by-channel convolution module, the output end of the 3×3 channel-by-channel convolution module is connected to the second input end of the addition module, the output end of the addition module is connected to the input end of the inverse Fourier transform module, and the output end of the inverse Fourier transform module is the output end of the entire module;

[0034] The probability feature map P of the garbage classification functional area of ​​the basic block of the city to be tested is used as the input of the scene feature enhancement module EPB, which is connected to the input ends of the first 1×1 convolution module, the third 1×1 convolution module and the fifth 1×1 convolution module respectively. The output of the first RFB module in the first branch is also connected to the input of the third 1×1 convolution module in the second branch, the output of the second 1×1 convolution module in the first branch is connected to the input of the second RFB module in the second branch, the output of the second RFB module in the second branch is connected to the input of the fifth 1×1 convolution module in the third branch, and the output of the fourth 1×1 convolution module in the second branch is connected to the input of the third RFB module in the third branch. The scene feature of the output of the entire scene feature enhancement module EPB is used express:

[0035]

[0036] in Includes three sizes and Matching feature map , =1, 2, 3;

[0037] is the output of the second 1×1 convolutional module, is the output of the fourth 1×1 convolutional module, is the output of the sixth 1×1 convolution module;

[0038] Step 4-3: and Perform AD fusion calculation: first use 1×1 convolution to Projection , the projection result and First, dot multiplication is performed, and then the deep learning activation function Sigmoid calculation is performed to obtain , and then and Perform dot multiplication to obtain combined pixel features and scene features The fusion features and express;

[0039] Step 4-4: The The detection results of informal garbage dumping sites in the basic blocks of the city to be tested are output through the non-maximum suppression algorithm NMS. The detection results include the suspicious spot sequence S1 of informal garbage dumping sites, the image detection confidence of informal garbage dumping sites , and the probability of garbage classification in image detection of informal garbage dumping sites , Representative The confidence level of image detection that the suspicious spots are informal garbage dumping sites. Representative The suspicious spot is The probability of image detection of informal garbage dumping sites, is the serial number of the garbage category label, The value can be any one of the five categories: domestic waste, construction waste, industrial waste, ecological waste, and mixed waste.

[0040] Further improved, the step 6 comprises:

[0041] Step 6-1: Extract the suspicious point image sequence S1 of the informal garbage dumping point according to the informal garbage dumping point image detection model in step 4, and further perform garbage detection using the point cloud detection model constructed in step 5 to obtain the point cloud detection confidence of each suspicious point image And point cloud detection category probability , Represents the first The confidence level of point cloud detection that the suspicious spots are informal garbage dumping points. Represents the first The first Point cloud detection probability of informal garbage dumping points, is the serial number of the garbage category label, The value of can be any one of the five categories: domestic waste, construction waste, industrial waste, ecological waste, and mixed waste;

[0042] Step 6-2: Fuse the results of the informal garbage dumping point image detection model and the point cloud detection model, and calculate the fusion probability of the suspicious point patch sequence S1 one by one. The calculation method is as follows:

[0043]

[0044] in, is the patch number in the suspicious patch sequence S1, The serial number of the garbage category described in the informal garbage dumping site, and are weight coefficients, which are set to 0.3 and 0.7 respectively. Then the maximum fusion probability is taken and the fusion detection confidence is output. , and obtain the final map of informal garbage dumping points and the corresponding garbage categories.

[0045] The technical solution adopted by the present invention to solve the above-mentioned further technical problems is: a computer-readable storage medium, which stores a computer program and can be read and executed by a processor, and is characterized in that: when the computer program is executed by the processor, it implements the above-mentioned informal garbage dumping site detection method.

[0046] Compared with the prior art, the advantages of the present invention are:

[0047] (1) It provides an end-to-end urban waste detection process, including the entire process from data collection, waste classification functional area division, image detection, point cloud recognition to multi-dimensional fusion, providing a comprehensive solution for urban management and waste disposal;

[0048] (2) By integrating multi-source data such as urban POI distribution, aerial images, and 3D point clouds, we can improve our overall understanding of the urban environment and achieve high-precision extraction of informal garbage dumping sites.

[0049] (3) Through image-assisted point cloud recognition, the processing speed is improved while ensuring the accuracy of the extraction results, and it has the high efficiency and real-time performance required in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flow chart of a method for detecting informal garbage dumping sites in an embodiment of the present invention.

[0051] Figure 2 4 is a functional module connection diagram of the informal garbage dumping site image detection model in an embodiment of the present invention.

[0052] Figure 3 It is a functional module connection diagram of the scene feature enhancement module EPB in an embodiment of the present invention.

[0053] Figure 4 4 is a connection diagram of the RFB module in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention is further described in detail below with reference to the accompanying drawings.

[0055] This embodiment provides a method for detecting informal garbage dumping sites, which is used to detect informal garbage dumping sites in a city block to be detected. Figure 1 As shown, it includes the following steps:

[0056] Step 1: construct three-dimensional point cloud data and orthophotos of the city blocks to be tested based on aerial images, and obtain road network data and point of interest data of the city blocks to be tested;

[0057] Generally, drone oblique photography technology can be used to obtain high-resolution aerial images of the city blocks to be tested, and then real-life 3D modeling software can be used to process the aerial images to construct 3D point cloud data and orthophotos of the city blocks to be tested; road network data of the city blocks to be tested can be obtained based on the open street map, and points of interest of the city blocks to be tested can be obtained based on Baidu map POI;

[0058] Step 2: Classify and annotate the three-dimensional point cloud data of the urban blocks to be tested and the informal garbage dumping points in the orthophotos through visual interpretation and manual interpretation, and generate garbage image detection sample sets and garbage point cloud detection sample sets, including:

[0059] Step 2-1, use the geographic information application software arcgis to mark the circumscribed rectangle of the informal garbage dumping site on the orthophoto of the city block to be tested, and assign a garbage category label to each informal garbage dumping site based on the visual features, contextual information and possible garbage sources of the informal garbage dumping site. The garbage category labels include five categories: domestic garbage, construction garbage, industrial garbage, ecological garbage, and mixed garbage; crop the orthophoto of the city block to be tested to a size of 512×512 using a sliding window method, and use a .json file to store the location and category information of the informal garbage dumping site;

[0060] Step 2-2: Use the SementicKITTI point cloud annotation tool to mark the point cloud set of each informal garbage dumping point in the three-dimensional point cloud data of the city block to be tested, and assign the corresponding garbage category label;

[0061] Step 3: Garbage classification functional area division: Segment and extract the basic blocks of the city to be tested based on the road network data of the city blocks to be tested, and calculate the interest point feature map of each basic block; extract the natural landscape feature map by classifying the objects through remote sensing interpretation of orthophotos; then use the interest point feature map and natural landscape feature map to perform cluster analysis on the basic blocks of the city to be tested, and obtain the probability feature map of garbage classification functional areas of the city blocks to be tested , and generate garbage classification functional areas of basic blocks in the city to be tested;

[0062] Specifically include:

[0063] Step 3-1, in the road network data of the city block to be tested, expressways, trunk roads, main roads, secondary roads, third-level roads with a length greater than 150m and unclassified roads are selected as the basic road network data of the city block to be tested;

[0064] Step 3-2: In the basic road network data of the city block to be tested obtained in step 3-1, the two-lane roads are merged into single-lane roads, and the road spaces including expressways, trunk roads, main roads, secondary roads, and branch roads are removed from the basic road network data of the city block to be tested to obtain independently distributed plot units, and then the merged area is less than 30,000m 2 The basic block distribution map of the city to be tested is obtained by

[0065] Step 3-3: The correlation between the category names of the points of interest in the city blocks to be tested and the garbage categories, reclassify the points of interest in the city blocks to be tested into five categories: residential, commercial, industrial, ecological and transportation, and calculate the feature map of the points of interest in each basic block. :

[0066]

[0067] in, is the serial number of the point of interest, is the block number, For the The first The number of interest points of the class, For the The number of points of interest of all types in the block, For the The first The sum of the areas occupied by the points of interest of the same type, For the The area of ​​the block;

[0068] Step 3-4: Use the land feature classification method of remote sensing interpretation to classify the basic block distribution map of the city to be tested. The land feature classification categories include farmland, forest and grassland, buildings, roads, bare land, water areas and others. Extract the natural landscape feature map from the basic block distribution map of the city to be tested. :

[0069]

[0070] in is the classification number of the land feature, For the The first The number of patches classified as land features, For the The total number of patches in the block, For the The first The sum of the areas occupied by the elements of this class, For the Mean area of ​​the feature classification; For the The first The sum of the perimeters of the feature classifications, For the Mean perimeter of the feature classification;

[0071] Step 3-5: Use the feature map of interest points obtained in step 3-3 And the natural landscape feature map obtained in steps 3-4 The Gaussian mixture model is used to perform cluster analysis on the basic block distribution map of the city to be tested. The cluster analysis method adopts the conventional method in the prior art to obtain the probability characteristic map of the garbage classification functional area of ​​the city block to be tested. ;

[0072] Then calculate the probability characteristic map of garbage classification functional areas The extreme value of the garbage classification functional area category of the basic block of the city to be tested is obtained , the categories of garbage classification functional areas include residential areas, commercial areas, industrial areas, ecological areas, mixed functional areas, and other areas:

[0073]

[0074] in, It is the category number of the garbage classification functional area. The value of can be any one of residential area, commercial area, industrial area, ecological area, mixed function area, and other areas. Indicates Blocks belong to category The probability of, argmax means calculating the extreme value in all garbage classification functional area categories;

[0075] Step 4: Use the garbage image detection sample set generated in step 2 to train the target detection model, extract the pixel features in the garbage image, use the garbage classification functional area probability feature map P generated in step 3 to extract the scene features, and then fuse the pixel features and scene features to build an image detection model for informal garbage dumping points in the basic blocks of the city to be tested, see Figure 2 The functional module connection diagram of the informal garbage dumping point image detection model shown in the figure is as follows:

[0076] Step 4-1: Use the Yolov8 target detection model to take the junk image detection sample set generated in step 2 as input data, extract pixel features from the junk image, and represent the original image in the junk image detection sample set as , corresponding to the pixel features of the garbage image It is expressed as:

[0077]

[0078] in, Contains features of 3 sizes, and the features of each size are expressed as , m=1,2,3;

[0079] Step 4-2, construct a scene feature enhancement module EPB, the scene feature enhancement module EPB includes three parallel branches - the first branch, the second branch and the third branch, wherein the first branch includes the first 1×1 convolution module, the first RFB module and the second 1×1 convolution module connected in sequence, the second branch includes the third 1×1 convolution module, the second RFB module and the fourth 1×1 convolution module connected in sequence, and the third branch includes the fifth 1×1 convolution module, the third RFB module and the sixth 1×1 convolution module connected in sequence, see Figure 3 As shown; the first RFB module, the second RFB module and the third RFB module have the same structure, and all include a Fourier transform module, a 3×3 channel-by-channel convolution module, an addition module, and an inverse Fourier transform module, see Figure 4 As shown, the input end of the Fourier transform module is the input end of the entire module, the output end of the Fourier transform module is connected to the first input end of the addition module, the output end of the Fourier transform module is also connected to the input end of the 3×3 channel-by-channel convolution module, the output end of the 3×3 channel-by-channel convolution module is connected to the second input end of the addition module, the output end of the addition module is connected to the input end of the inverse Fourier transform module, and the output end of the inverse Fourier transform module is the output end of the entire module;

[0080] The probability feature map P of the garbage classification functional area of ​​the basic block of the city to be tested is used as the input of the scene feature enhancement module EPB, which is connected to the input ends of the first 1×1 convolution module, the third 1×1 convolution module and the fifth 1×1 convolution module respectively. The output of the first RFB module in the first branch is also connected to the input of the third 1×1 convolution module in the second branch, the output of the second 1×1 convolution module in the first branch is connected to the input of the second RFB module in the second branch, the output of the second RFB module in the second branch is connected to the input of the fifth 1×1 convolution module in the third branch, and the output of the fourth 1×1 convolution module in the second branch is connected to the input of the third RFB module in the third branch. The scene feature of the output of the entire scene feature enhancement module EPB is used express:

[0081]

[0082] in Includes three sizes and Matching feature map , =1, 2, 3;

[0083] is the output of the second 1×1 convolutional module, is the output of the fourth 1×1 convolutional module, is the output of the sixth 1×1 convolution module;

[0084] Step 4-3: and Perform AD fusion calculation: First use a 1×1 convolution module to Projection , the projection result and First, dot multiplication is performed, and then the deep learning activation function Sigmoid calculation is performed to obtain , and then and Perform dot multiplication to obtain combined pixel features and scene features The fusion features and express;

[0085] Step 4-4: The The detection results of informal garbage dumping sites in the basic blocks of the city to be tested are output through the non-maximum suppression algorithm NMS. The detection results include the suspicious spot sequence S1 of informal garbage dumping sites, the image detection confidence of informal garbage dumping sites , and the probability of garbage classification in image detection of informal garbage dumping sites , Representative The confidence level of image detection that the suspicious spots are informal garbage dumping sites. Representative The suspicious spot is The probability of image detection of informal garbage dumping sites, is the serial number of the garbage category label, The value of can be any one of the five categories: domestic waste, construction waste, industrial waste, ecological waste, and mixed waste;

[0086] Step 5: construct a point cloud detection model for the basic blocks of the city to be tested, and use the garbage point cloud detection sample set generated in step 2 to train the point cloud detection model;

[0087] A 3D color point cloud classification model is constructed based on Point Transformer. The garbage point cloud detection sample set generated in step 2 is used as input data to train the 3D color point cloud classification model. The point information in the garbage point cloud detection sample set is ,in is the corresponding coordinate information, is the color information of the corresponding point is the garbage category of the point; predict and output the category label and probability distribution of each point; the method used to construct the point cloud detection model of the basic blocks of the city to be tested is a conventional method in the prior art and will not be described in detail here;

[0088] Step 6: Extract the suspicious spot sequence S1 of the informal garbage dumping point according to the informal garbage dumping point image detection model in step 4, and then use the point cloud detection model constructed in step 5 to further detect the suspicious spot sequence S1, and output the recognition result through weighted fusion, which specifically includes:

[0089] Step 6-1: Extract the suspicious point image sequence S1 of the informal garbage dumping point according to the informal garbage dumping point image detection model in step 4, and further perform garbage detection using the point cloud detection model constructed in step 5 to obtain the point cloud detection confidence of each suspicious point image And point cloud detection category probability , Represents the first The confidence level of point cloud detection that the suspicious spots are informal garbage dumping points. Represents the first The first Point cloud detection probability of informal garbage dumping points, is the serial number of the garbage category label, The value of can be any one of the five categories: domestic waste, construction waste, industrial waste, ecological waste, and mixed waste;

[0090] Step 6-2: Fuse the results of the informal garbage dumping point image detection model and the point cloud detection model, and calculate the fusion probability of the suspicious point patch sequence S1 one by one. The calculation method is as follows:

[0091]

[0092] in, is the patch number in the suspicious patch sequence S1, The serial number of the garbage category described in the informal garbage dumping site, and are weight coefficients, which are set to 0.3 and 0.7 respectively. Then the maximum fusion probability is taken and the fusion detection confidence is output. , and obtain the final map of informal garbage dumping points and the corresponding garbage categories.

[0093] Finally, this embodiment also provides a computer-readable storage medium, which stores a computer program and can be read and executed by a processor. When the above computer program is executed by the processor, the above-mentioned informal garbage dumping site detection method is implemented.

Claims

1. A method for detecting informal garbage dumping sites, used to detect informal garbage dumping sites in a city block to be detected, characterized in that The steps include: Step 1: construct three-dimensional point cloud data and orthophotos of the city blocks to be tested based on aerial images, and obtain road network data and point of interest data of the city blocks to be tested; Step 2: Classify and annotate the three-dimensional point cloud data of the urban blocks to be tested and the informal garbage dumping points in the orthophotos by means of visual interpretation and manual interpretation, and generate a garbage image detection sample set and a garbage point cloud detection sample set; Step 3: Segment and extract the basic blocks of the city to be tested based on the road network data of the city blocks to be tested, and calculate the feature map of the points of interest of each basic block; classify the objects through remote sensing interpretation of the orthophoto, and extract the feature map of the natural landscape; The basic blocks of the city to be tested are clustered using the feature map of points of interest and the feature map of natural landscapes to obtain the probability feature map P of the garbage classification functional areas of the city blocks to be tested, which specifically includes: Step 3-1, in the road network data of the city block to be tested, expressways, trunk roads, main roads, secondary roads, third-level roads with a length greater than 150m and unclassified roads are selected as the basic road network data of the city block to be tested; Step 3-2: In the basic road network data of the city block to be tested obtained in step 3-1, the two-lane roads are merged into single-lane roads, and the road spaces including expressways, trunk roads, main roads, secondary roads, and branch roads are removed from the basic road network data of the city block to be tested to obtain independently distributed plot units, and then the merged area is less than 30,000m 2 of plots to obtain the basic block distribution map of the city to be tested; Step 3-3: The correlation between the category names of the points of interest in the city blocks to be tested and the garbage categories, reclassify the points of interest in the city blocks to be tested into five categories: residential, commercial, industrial, ecological and transportation, and calculate the feature map of the points of interest for each basic block WST ij : Among them, i is the number of the point of interest, j is the number of the block, and N ij is the number of interest points of type i in the jth block, N j is the number of all types of points of interest in the jth block, S ij is the sum of the areas occupied by the i-th type of interest points in the j-th block, S j is the area of ​​the jth block; Step 3-4: Use the land feature classification method of remote sensing interpretation to classify the basic block distribution map of the city to be tested. The land feature classification categories include farmland, forest and grassland, buildings, roads, bare land, water areas and others, a total of seven categories, and extract the natural landscape feature map NST from the basic block distribution map of the city to be tested kj ; Step 3-5: Use the interest point feature map WST obtained in step 3-3 ij And the natural landscape feature map NST obtained in step 3-4 kj , a Gaussian mixture model is used to perform cluster analysis on the basic block distribution map of the city to be tested, and a probability characteristic map P of the garbage classification functional area of ​​the city blocks to be tested is obtained; Then calculate the extreme value of the probability characteristic map P of the garbage classification functional area to obtain the garbage classification functional area category C of the basic block of the city to be tested j , the categories of garbage classification functional areas include residential areas, commercial areas, industrial areas, ecological areas, mixed functional areas, and other areas: C j =argmaxP(Class j =a) Among them, a is the category number of the garbage classification functional area, and the value of a can be any one of residential area, commercial area, industrial area, ecological area, mixed functional area, and other areas. j =a) represents the probability that the jth block belongs to category a, and argmax represents the calculation of extreme values ​​among all garbage classification functional area categories; Step 4: Use the garbage image detection sample set generated in step 2 to train the Yolov8 target detection model, extract pixel features in the garbage image, use the garbage classification functional area probability feature map P generated in step 3 to extract scene features, and then fuse pixel features and scene features to construct an image detection model for informal garbage dumping points in the basic blocks of the city to be tested; Step 5: construct a point cloud detection model for the basic blocks of the city to be tested, and use the garbage point cloud detection sample set generated in step 2 to train the point cloud detection model; Step 6: Extract the suspicious spot sequence S1 of informal garbage dumping points according to the image detection model of the basic blocks of the city to be tested in step 4, and then further detect the suspicious spot sequence S1 using the point cloud detection model constructed in step 5, and output the identification result of informal garbage dumping points through weighted fusion.

2. The method for detecting informal garbage dumping sites according to claim 1, characterized in that: In the step 1, the drone oblique photography technology is used to obtain high-resolution aerial images of the city blocks to be tested, and then the aerial images are processed using real-scene 3D modeling software to construct 3D point cloud data and orthophotos of the city blocks to be tested; the road network data of the city blocks to be tested is obtained based on the open street map, and the points of interest of the city blocks to be tested are obtained based on the Baidu map POI.

3. The method for detecting informal garbage dumping sites according to claim 1, characterized in that: The step 2 comprises: Step 2-1, use the geographic information application software arcgis to mark the circumscribed rectangle of the informal garbage dumping site on the orthophoto of the city block to be tested, and assign a garbage category label to each informal garbage dumping site based on the visual features, contextual information and possible garbage sources of the informal garbage dumping site. The garbage category labels include five categories: domestic garbage, construction garbage, industrial garbage, ecological garbage, and mixed garbage; crop the orthophoto of the city block to be tested to a size of 512×512 using a sliding window method, and use a .json file to store the location and category information of the informal garbage dumping site; Step 2-2: Use the SementicKITTI point cloud annotation tool to mark the point cloud set of each informal garbage dumping point in the three-dimensional point cloud data of the urban block to be tested, and assign corresponding garbage category labels.

4. The method for detecting an informal garbage dumping site according to claim 1, characterized in that: The step 4 specifically includes: Step 4-1: Use the Yolov8 target detection model, take the junk image detection sample set generated in step 2 as input data, extract the pixel features in the junk image, represent the original image in the junk image detection sample set as OI, and the pixel feature F corresponding to the junk image pixel It is expressed as: F pixel =YOLO(HI) Among them, F pixel Contains features of 3 sizes, each size feature is represented by F pixel (m), m = 1, 2, 3; Step 4-2, construct a scene feature enhancement module EPB, the scene feature enhancement module EPB includes three parallel branches - a first branch, a second branch and a third branch, wherein the first branch includes a first 1×1 convolution module, a first RFB module and a second 1×1 convolution module connected in sequence, the second branch includes a third 1×1 convolution module, a second RFB module and a fourth 1×1 convolution module connected in sequence, the third branch includes a fifth 1×1 convolution module, a third RFB module and a sixth 1×1 convolution module connected in sequence, the first RFB module, the second RFB module and the third RFB module The modules have the same structure, and all include a Fourier transform module, a 3×3 channel-by-channel convolution module, an addition module, and an inverse Fourier transform module, wherein the input end of the Fourier transform module is the input end of the entire module, the output end of the Fourier transform module is connected to the first input end of the addition module, the output end of the Fourier transform module is also connected to the input end of the 3×3 channel-by-channel convolution module, the output end of the 3×3 channel-by-channel convolution module is connected to the second input end of the addition module, the output end of the addition module is connected to the input end of the inverse Fourier transform module, and the output end of the inverse Fourier transform module is the output end of the entire module; The probability feature map P of the garbage classification functional area of ​​the basic block of the city to be tested is used as the input of the scene feature enhancement module EPB, which is connected to the input ends of the first 1×1 convolution module, the third 1×1 convolution module and the fifth 1×1 convolution module respectively. The output of the first RFB module in the first branch is also connected to the input of the third 1×1 convolution module in the second branch, the output of the second 1×1 convolution module in the first branch is connected to the input of the second RFB module in the second branch, the output of the second RFB module in the second branch is connected to the input of the fifth 1×1 convolution module in the third branch, and the output of the fourth 1×1 convolution module in the second branch is connected to the input of the third RFB module in the third branch. The scene features of the output of the entire scene feature enhancement module EPB are represented by F block express: F block =EPB(P) where F block Includes three sizes with F pixel Matched feature map F block (n), n = 1, 2, 3; F block (1) is the output of the second 1×1 convolutional module, F block (2) is the output of the fourth 1×1 convolutional module, F block (3) is the output of the sixth 1×1 convolution module; Step 4-3, F pixel and F block Perform AD fusion calculation: first use 1×1 convolution to F pixel Projection to get F′ pixel , the projection result F′ pixel With F block First perform the dot multiplication operation, and then perform the deep learning activation function Sigmoid calculation to get F′ final , and then F pixel and F′ final Perform a dot multiplication operation to obtain the combined pixel feature F pixel and scene feature F block The fusion features and F final express; Step 4-4: Add the F obtained in step 4-3 final The detection results of informal garbage dumping sites in the basic blocks of the city to be tested are output through the non-maximum suppression algorithm NMS. The detection results include the suspicious spot sequence S1 of informal garbage dumping sites and the image detection confidence Conf of informal garbage dumping sites. img , and the probability of garbage classification P for image detection of informal garbage dumping sites img , Conf img (b) represents the confidence level of image detection that the b-th suspicious spot is an informal garbage dumping site, P img (b)(c) represents the image detection probability that the b-th suspicious spot is the c-th type of informal garbage dumping site. c is the serial number of the garbage category label, and the value of c can be any one of the five categories: domestic waste, construction waste, industrial waste, ecological waste, and mixed waste.

5. The method for detecting informal garbage dumping sites according to claim 4, characterized in that: The step 6 comprises: Step 6-1: Extract the suspicious spot sequence S1 of the informal garbage dumping point according to the image detection model of the informal garbage dumping point in step 4, and further perform garbage detection using the point cloud detection model constructed in step 5 to obtain the point cloud detection confidence Conf of each suspicious spot. point And point cloud detection category probability P point , Conf point (b) represents the point cloud detection confidence that the b-th suspicious point is an informal garbage dumping point, P point (b)(c) represents the point cloud detection probability that the bth spot is the cth type of informal garbage dumping site obtained by point cloud detection, c is the serial number of the garbage category label, and the value of c is any one of the five categories of domestic garbage, construction garbage, industrial garbage, ecological garbage, and mixed garbage; Step 6-2: Fuse the results of the informal garbage dumping point image detection model and the point cloud detection model, and calculate the fusion probability of the suspicious point patch sequence S1 one by one. The calculation method is as follows: P fused (b)(c)=α·Conf img (b)·P img (b)(c)+β·Conf point (b)·P point (b)(c) Among them, b is the number of the suspicious spot sequence S1, c is the number of the garbage category described in the informal garbage dumping point, α and β are weight coefficients, which are set to 0.3 and 0.7 respectively, and then the maximum fusion probability is taken to output the fusion detection confidence P fused , and obtain the final map of informal garbage dumping points and the corresponding garbage categories.

6. A computer-readable storage medium storing a computer program and capable of being read and executed by a processor, characterized in that: When the computer program is executed by a processor, the method for detecting an informal garbage dumping site as claimed in any one of claims 1 to 5 is implemented.

Images