Intelligent identification system for abandoned deposits in built-up area based on multi-source remote sensing image and unmanned aerial vehicle patrol

Abstract

The invention discloses a built-up area waste deposit intelligent identification system based on multi-source remote sensing images and unmanned aerial vehicle patrol, and the system comprises a data acquisition module, a feature extraction module, an intelligent identification module and a result verification module. Meanwhile, automatic detection and classification of the waste deposits in the built-up area are achieved through a deep learning algorithm, unmanned aerial vehicle patrol data are used for conducting real-time correction and supplement on remote sensing recognition results, the recognition precision and timeliness are improved, the situation of the waste deposits in the urban environment can be efficiently and accurately recognized, and the recognition efficiency is improved. The system realizes intelligent and automatic monitoring of waste accumulation in the urban environment, provides data support and scientific basis for urban management and ecological management, has the advantages of high precision, wide coverage and dynamic updating, and is beneficial to scientific management and improvement of the urban environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of urban construction management, in particular to a built-up area abandoned accumulation intelligent identification system based on multi-source remote sensing images and unmanned aerial vehicle patrol. BACKGROUND

[0002] The abandoned accumulation in the built-up area of the city mainly includes construction waste, household garbage, industrial waste residue and other solid waste. If these materials are randomly stacked without standard disposal, not only the city environment is affected and the traffic is hindered, but also soil and water pollution and even safety hazards may be caused. In recent years, with the promotion of waste-free city construction, many places have strengthened the whole-chain management of abandoned accumulation. Common abandoned accumulation includes construction waste, large household garbage and mixed garbage piles. In the traditional identification of abandoned accumulation, single satellite or manual patrol is relied on, which has the disadvantages of incomplete coverage, poor timeliness, and is easily affected by weather, and the actual image recognition accuracy is low. At the same time, satellite patrol has a wide area and can be identified at a macro angle, but cannot be identified with more subtle precision. Images and geographic data coordinates and scales obtained at different times and from different sources cannot be directly analyzed jointly, resulting in great difficulty in actual monitoring. SUMMARY

[0003] The present application provides a built-up area abandoned accumulation intelligent identification system based on multi-source remote sensing images and unmanned aerial vehicle patrol, which can effectively solve the problem of relying on single satellite or manual patrol in the traditional identification of abandoned accumulation, which has the disadvantages of incomplete coverage, poor timeliness, and is easily affected by weather, and the actual image recognition accuracy is low. At the same time, satellite patrol has a wide area and can be identified at a macro angle, but cannot be identified with more subtle precision. Images and geographic data coordinates and scales obtained at different times and from different sources cannot be directly analyzed jointly, resulting in great difficulty in actual monitoring.

[0004] To achieve the above purpose, the present application provides the following technical scheme: a built-up area abandoned accumulation intelligent identification system based on multi-source remote sensing images and unmanned aerial vehicle patrol, which realizes intelligent and automatic monitoring of abandoned accumulation in urban environment, including data acquisition module, feature extraction module, intelligent identification module and result verification module. The data acquisition module is responsible for integrating multi-source remote sensing data and geographic information, and the feature extraction module is responsible for extracting key features that distinguish abandoned accumulation from other ground objects from multi-source data. The intelligent identification module realizes automatic identification and classification of abandoned accumulation based on deep learning algorithm, and the result verification module identifies the reliability of the result by multi-method evaluation and corrects it in real time by using unmanned aerial vehicle patrol data.

[0005] According to the technical scheme, the data acquisition module comprises a satellite image acquisition sub-module, a UAV aerial image acquisition sub-module and a geographic information data acquisition sub-module; The satellite image acquisition sub-module uses optical satellites and synthetic aperture radars in combination, optical satellite images are used to capture color and texture features of the waste accumulation, and radar imaging is used to penetrate thin clouds and vegetation shelter; Meanwhile, satellite images are regularly collected according to the characteristics of the urban built-up area on a quarterly basis, and are dynamically collected on demand in combination with emergency requirements; The UAV aerial image acquisition sub-module captures aerial images by carrying a visible light digital camera on a multi-rotor UAV; The geographic information data acquisition sub-module acquires basic geographic data and administrative boundary by accessing city DEM, DSM and land use status map, and marks potential accumulation high-risk areas by integrating city management complaint records, environmental sanitation operation trajectories and construction site supervision data.

[0006] According to the technical scheme, the UAV aerial image acquisition sub-module needs to design low-altitude flight routes for suspected areas and management key areas identified by satellites during aerial image capturing; The suspected areas and management key areas are finely captured at low altitude according to the designed flight routes, the images captured by the UAV are automatically associated with GPS positioning information, and are spatio-temporally matched with satellite image data; The registration error is evaluated: Wherein: : Position of control point in reference coordinate; Position of control point after registration; N: Number of control points.

[0007] According to the technical scheme, after the satellite image data and the UAV aerial image data are acquired, the data acquisition module needs to eliminate the influence of sensor noise and atmospheric scattering on the satellite image data and the aerial image data based on radiation correction, and unify the coordinate system and the spatial resolution of the multi-source images through geometric fine correction based on geographic information data; In order to weaken the influence of sensor gain difference and atmospheric scattering on pixel response, linear radiation normalization processing can be performed on each band: Wherein: : Pixel value of original image in band b; : Corrected pixel value; , : coefficients obtained by calibrating with the same sensor or statistically fitting the same scene.

[0008] According to the above technical solution, the feature extraction module includes a spectral feature extraction submodule, a spatial morphology feature extraction submodule, and a time sequence feature extraction submodule.

[0009] According to the above technical solution, the feature extraction module also needs to fuse multi-modal image data, wherein satellite image data extracts macro-distribution features, unmanned aerial vehicle image data extracts micro-detail features, and the low-resolution features of satellite image data are associated with the high-resolution local features of unmanned aerial vehicles. At the same time, the relationship between the target and the surrounding geographical features is analyzed in combination with geographical data information.

[0010] According to the above technical solution, the intelligent identification module includes a model architecture design submodule and a multi-source data fusion submodule. The model architecture design submodule uses an improved YOLOv8, optimizes the anchor box size for small garbage piles, introduces an attention mechanism to focus on key areas, and inputs the image after multi-source data fusion. And embed a lightweight CNN in the detection box to output the classification result.

[0011] According to the above technical solution, in the early fusion process, the satellite and unmanned aerial vehicle images are aligned, stacked by band, and used as model input. In the late fusion process, two detection models are trained using satellite images and unmanned aerial vehicle images respectively, and the results are fused by weighted voting.

[0012] According to the above technical solution, the result verification module includes a quantitative evaluation submodule, an on-site verification and manual verification submodule, an unmanned aerial vehicle real-time correction submodule, and a dynamic update and visualization submodule. The quantitative evaluation submodule calculates the precision, recall, and average precision mean based on the verification set annotated by artificial labeling, evaluates the overall performance of the model, and calculates the types of false positives and the reasons for missing detection through a confusion matrix.

[0013] According to the above technical solution, the unmanned aerial vehicle real-time correction submodule triggers the correction process when the satellite identification result conflicts with the unmanned aerial vehicle patrol data, and automatically dispatches the unmanned aerial vehicle for close inspection.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: By integrating high-resolution satellite imagery, UAV aerial imagery, and geographic information data, the present invention solves the contradiction of not being able to simultaneously take into account macroscopic distribution features and microscopic detail features through the combined application of satellite and UAV. Furthermore, by combining optical imagery with SAR imagery, it achieves monitoring capabilities regardless of weather conditions. By combining wide-area satellite surveys with local detailed UAV surveys, it constructs a more microscopic observation system, facilitating the acquisition of refined data information. Simultaneously, deep learning algorithms are used to automatically detect and classify waste accumulation within the built-up area. UAV patrol data is used to correct and supplement the remote sensing identification results in real time, improving the accuracy and timeliness of identification. It can efficiently and accurately identify the situation of waste accumulation in the urban environment, realize intelligent and automated monitoring of waste accumulation in the urban environment, provide data support and scientific basis for urban management and ecological governance, and has the advantages of high precision, wide coverage and dynamic updates, which helps to scientifically manage and improve the urban environment. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0016] In the attached diagram: Figure 1 This is an architecture diagram of the intelligent recognition system of the present invention. Detailed Implementation

[0017] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0018] Example: Figure 1 As shown, the present invention provides a technical solution: an intelligent identification system for waste accumulation in built-up areas based on multi-source remote sensing images and UAV patrols. This system enables intelligent and automated monitoring of waste accumulation in the urban environment, providing a scientific basis for urban management and ecological governance. The system includes a data acquisition module, a feature extraction module, an intelligent identification module, and a result verification module. The data acquisition module is responsible for integrating multi-source remote sensing data and geographic information to provide high-quality input for subsequent processing. The feature extraction module is responsible for extracting key features from multi-source data to distinguish abandoned deposits from other land features, supporting subsequent intelligent identification. The intelligent identification module uses deep learning algorithms to realize the automatic identification, detection and classification of abandoned deposits. The result verification module uses multiple evaluation methods to identify the reliability of the results and uses UAV patrol data for real-time correction.

[0019] Based on the above technical scheme, the data acquisition module includes a satellite image acquisition submodule, an unmanned aerial vehicle aerial image acquisition submodule, and a geographic information data acquisition submodule; The satellite image acquisition submodule uses a high-resolution optical satellite combined with a synthetic aperture radar, taking into account the visible light and microwave characteristics, and the optical satellite image is used to capture the color and texture features of the waste accumulation, and the radar imaging is used to penetrate thin clouds and vegetation barriers to identify hidden accumulations. At the same time, satellite images are regularly collected according to the characteristics of the urban built-up area on a quarterly basis, and are dynamically collected on demand in combination with emergency needs. The unmanned aerial vehicle aerial image acquisition submodule uses a visible light digital camera mounted on a multi-rotor unmanned aerial vehicle to take aerial images. The geographic information data acquisition submodule acquires basic geographic data and administrative boundary by accessing city DEM, DSM, and land use status map, EM is a digital elevation model with a resolution of 5m, DSM is a digital surface model, and land use status map is used to distinguish buildings, roads, and green spaces. At the same time, city management complaint records, sanitation operation trajectories, and construction site supervision data are integrated to mark potential accumulation high-risk areas. The complaint records include location and type, the sanitation operation trajectories are obtained through the GPS of the garbage collection truck, and the construction site supervision data includes construction range and duration.

[0020] Based on the above technical scheme, the unmanned aerial vehicle aerial image acquisition submodule needs to design low-altitude flight routes for suspected areas identified by satellites and management key areas during aerial image acquisition. Suspected areas refer to high-confidence candidate areas that need to be refined, and management key areas refer to construction sites and demolition areas. The flight height of the low-altitude flight route is 120 meters. The suspected areas and management key areas are finely photographed according to the designed flight route to ensure the capture of details in the aerial images. The images collected by the unmanned aerial vehicle automatically associate with GPS positioning information and are spatio-temporally matched with satellite image data. The registration error is evaluated in coordination: Wherein: : Position of control point in reference coordinate; Position of control point after registration; N: Number of control points.

[0021] Based on the above technical scheme, after acquiring satellite image data and unmanned aerial vehicle aerial image data, the data acquisition module needs to eliminate the influence of sensor noise and atmospheric scattering on satellite image data and aerial image data based on radiation correction, and unify the coordinate system and spatial resolution of multiple source images through geometric fine correction based on geographic information data. In order to weaken the influence of sensor gain difference and atmospheric scattering on the pixel response, linear radiometric normalization processing can be performed on each band: Wherein: : the pixel value of the original image in the band b; : the corrected pixel value; , : the coefficient obtained by calibration of the same sensor or statistical fitting of the same scene.

[0022] At the same time, the local image of the unmanned aerial vehicle is spatially aligned with the global image of the satellite to generate a seamless image base map.

[0023] Based on the above technical scheme, the feature extraction module includes a spectral feature extraction submodule, a spatial morphological feature extraction submodule, and a time sequence feature extraction submodule. The spectral feature extraction submodule distinguishes between accumulated materials and natural features by calculating normalized vegetation index, normalized building index, and brightness value for satellite and unmanned aerial vehicle images. The normalized vegetation index (NDVI) distinguishes between vegetation and accumulated materials, the normalized building index (NDBI) distinguishes between hardened ground, the brightness value distinguishes between the reflection differences of metal and plastic materials, and the red edge band is used to detect hidden garbage under vegetation cover. The spatial morphological feature extraction submodule calculates contrast, correlation, and entropy value through the gray level co-occurrence matrix (GLCM) to capture the roughness of the garbage surface. The contrast is specifically the chaotic texture of the accumulated material surface, and the entropy value reflects the roughness of the accumulated material surface. The area, perimeter, and aspect ratio are extracted through contour analysis, and the distance between the accumulated material and the road or construction site is calculated in combination with geographic information data to determine whether it is transported and discarded garbage. The time sequence feature extraction submodule performs change detection on historical satellite images and current images of the same area to identify newly added abnormal areas. At the same time, the same area is regularly photographed once a month to track the expansion and reduction trend of the accumulated material, which assists in determining temporary and long-term garbage accumulation.

[0024] Based on the above technical scheme, the feature extraction module also needs to fuse multi-modal image data. The satellite image data extracts macroscopic distribution features, the unmanned aerial vehicle image data extracts microscopic detail features, the low-resolution features of the satellite image data are associated with the high-resolution local features of the unmanned aerial vehicle, and the detection capability of small garbage piles is improved. At the same time, the relationship between the target and the surrounding features is analyzed in combination with geographic data information. Specifically, the accumulated material near the road or construction site is construction waste, and the accumulated material near the residential area is household garbage.

[0025] Based on the above technical solution, the intelligent identification module includes a model architecture design submodule and a multi-source data fusion submodule; The model architecture design submodule adopts an improved YOLOv8, optimizes the anchor box size for small garbage piles, introduces an attention mechanism to focus on key areas, and the input is the multi-source data fused image. The attention mechanism is CBAM, and the multi-source data fused image is a satellite plus UAV registered image; And a lightweight CNN is embedded in the detection box to output the classification result. The CNN is MobileNetV3, and the classification result includes construction waste, household waste, industrial waste, and mixed waste.

[0026] Based on the above technical solution, in the early fusion process, after aligning the satellite and UAV images, the multi-source data fusion submodule stacks them by band as the model input, retaining the original information; In the late fusion process, two detection models are trained using satellite images and UAV images respectively, and the results are fused by weighted voting to reduce the false detection rate of a single data source.

[0027] Based on the above technical solution, the result verification module includes a quantitative evaluation submodule, a field verification and manual checking submodule, a UAV real-time correction submodule, and a dynamic updating and visualization submodule; The quantitative evaluation submodule calculates the precision, recall, and average precision (mAP) based on the manually annotated validation set to evaluate the overall performance of the model, and uses the confusion matrix to count the false detection types and missed detection reasons to locate the weak areas of the model and optimize the feature extraction accordingly. False detection types include misjudging bare soil as garbage, and missed detection reasons include shadow obstruction; The field verification and manual checking submodule pushes the recognition results to the city management personnel APP, takes photos on site and labels the actual categories, uploads them, and automatically compares the system results with the manual annotations to generate an error report. At the same time, for areas with high confidence but sensitive to location, such as school surroundings, manual field measurement is organized to verify the volume and type of accumulated materials.

[0028] Based on the above technical solution, the UAV real-time correction submodule triggers the correction process when the satellite recognition result conflicts with the UAV patrol data. Specifically, the satellite recognition result shows that there is garbage in a certain area, while the UAV image shows that there is no garbage in that area. The UAV is automatically dispatched for close-up investigation, and the high-resolution result of the UAV image is used as the accurate value to adjust the feature extraction parameters of the satellite image in reverse; The dynamic updating and visualizing submodule supports backtracking of the evolution process of the accumulation according to the time axis by storing the time stamp, spatial coordinates and type of the identification result, and displays the distribution heat map and type proportion pie chart of the abandoned accumulation through the GIS map, and superimposes the urban management grid responsibility area to assist in accurate dispatching.

[0029] Finally, it should be noted that: the above only for the preferred examples of the present application, and not for the purpose of limiting the present application, although in the foregoing examples of the present application has been described in detail, for those skilled in the art, it still can be modified, or part of the technical features of the equivalent replacement of the technical solutions described in the foregoing embodiments. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. An intelligent identification system for abandoned debris in built-up areas based on multi-source remote sensing imagery and UAV patrols, characterized in that: To achieve intelligent and automated monitoring of waste accumulation in the urban environment, including a data acquisition module, a feature extraction module, an intelligent identification module, and a result verification module; The data acquisition module is responsible for integrating multi-source remote sensing data and geographic information, while the feature extraction module is responsible for extracting key features from the multi-source data that distinguish waste deposits from other land features. The intelligent identification module uses deep learning algorithms to automatically identify, detect, and classify waste accumulation. The result verification module uses multiple evaluation methods to identify the reliability of the results and uses drone patrol data for real-time correction.

2. The intelligent identification system for abandoned debris in built-up areas based on multi-source remote sensing imagery and UAV patrol as described in claim 1, characterized in that: The data acquisition module includes a satellite image acquisition submodule, a UAV aerial image acquisition submodule, and a geographic information data acquisition submodule; The satellite image acquisition submodule uses a combination of optical satellite and synthetic aperture radar. Optical satellite imagery is used to capture the color and texture features of waste piles, while radar imaging is used to penetrate thin clouds and vegetation obstructions. At the same time, satellite images are collected quarterly based on the characteristics of urban built-up areas, and dynamic collection is carried out as needed in conjunction with emergency requirements; The drone aerial image acquisition submodule acquires aerial images by mounting a visible light digital camera on a multi-rotor drone. The geographic information data acquisition submodule obtains basic geographic data and administrative boundaries by accessing urban DEM, DSM, and land use status maps. At the same time, it integrates urban management complaint records, sanitation operation trajectories, and construction site supervision data to mark areas with high potential accumulation of debris.

3. The intelligent identification system for abandoned debris in built-up areas based on multi-source remote sensing imagery and UAV patrols as described in claim 2, characterized in that: During the aerial image acquisition process, the UAV aerial image acquisition submodule needs to design low-altitude flight paths for suspected areas identified by satellite and key management areas. The drone will conduct low-altitude, high-resolution images of suspected and key management areas according to the designed flight path. The images collected by the drone will be automatically linked to GPS positioning information and spatiotemporally matched with satellite image data. Evaluation based on registration error: in: The position of the control point in the reference coordinate system; The position of the control points after registration; N: Number of control points.

4. The intelligent identification system for abandoned debris in built-up areas based on multi-source remote sensing imagery and UAV patrol as described in claim 2, characterized in that: After acquiring satellite imagery data and UAV aerial imagery data, the data acquisition module needs to eliminate the influence of sensor noise and atmospheric scattering on the satellite imagery data and aerial imagery data based on radiometric correction, and then unify the coordinate system and spatial resolution of the multi-source images based on geographic information data through geometric fine correction. To reduce the impact of sensor gain differences and atmospheric scattering on pixel response, linear radiative normalization can be performed on each band: in: : The pixel value of the original image in band b; Corrected pixel values; , : Coefficients obtained by calibration of the same sensor or statistical fitting of the same scene.

5. The intelligent identification system for abandoned debris in built-up areas based on multi-source remote sensing imagery and UAV patrol as described in claim 1, characterized in that: The feature extraction module includes a spectral feature extraction submodule, a spatial morphological feature extraction submodule, and a temporal feature extraction submodule.

6. The intelligent identification system for abandoned debris in built-up areas based on multi-source remote sensing imagery and UAV patrol as described in claim 5, characterized in that: The feature extraction module also needs to fuse multimodal image data, in which macroscopic distribution features are extracted from satellite image data and microscopic detail features are extracted from UAV image data, and the low-resolution features of satellite image data are correlated with the high-resolution local features of UAV. At the same time, geographic data is used to analyze the relationship between the target and surrounding features.

7. The intelligent identification system for abandoned debris in built-up areas based on multi-source remote sensing imagery and UAV patrol as described in claim 1, characterized in that: The intelligent recognition module includes a model architecture design submodule and a multi-source data fusion submodule; The model architecture design submodule adopts an improved YOLOv8, optimizes the anchor frame size for small garbage heaps, introduces an attention mechanism to focus on key areas, and the input is an image after multi-source data fusion. A lightweight CNN is embedded within the detection bounding box to output the classification results.

8. The intelligent identification system for abandoned debris in built-up areas based on multi-source remote sensing imagery and UAV patrol as described in claim 7, characterized in that: In the early fusion process, the multi-source data fusion submodule aligns satellite and UAV images, stacks them by band, and uses them as model input. In the late-stage fusion process, two detection models were trained using satellite imagery and UAV imagery respectively, and the results were fused by weighted voting.

9. The intelligent identification system for abandoned debris in built-up areas based on multi-source remote sensing imagery and UAV patrol as described in claim 1, characterized in that: The result verification module includes a quantitative evaluation submodule, an on-site verification and manual verification submodule, a UAV real-time correction submodule, and a dynamic update and visualization submodule. The quantitative evaluation submodule calculates precision, recall, and mean precision based on a manually labeled validation set to evaluate the overall performance of the model, and uses a confusion matrix to statistically analyze the types of false positives and the reasons for false negatives.

10. The intelligent identification system for abandoned debris in built-up areas based on multi-source remote sensing imagery and UAV patrol as described in claim 9, characterized in that: The real-time correction submodule for drones triggers a correction process when satellite identification results conflict with drone patrol data, automatically scheduling drones to conduct close-range detailed inspections.

Images