Garbage inspection method and system in garden area based on unmanned aerial vehicle

Abstract

The invention relates to the technical field of unmanned aerial vehicle inspection, and particularly discloses an unmanned aerial vehicle-based garbage inspection method and system in a garden area, and the method comprises the steps: obtaining environment data in the garden area, collecting the behavior mode of tourists in each area in the garden area, obtaining the feature data of obstacles in the garden area, the method comprises the following steps: determining an obstacle temporary no-fly zone in a garden area, carrying out three-dimensional modeling on the garden area through a multi-view RGB-D camera carried by an unmanned aerial vehicle, generating a garden area map, determining an optimal inspection path of the unmanned aerial vehicle, transmitting an inspection image of the unmanned aerial vehicle in the garden area to a data center for storage, and carrying out garbage detection and labeling in the garden area. And carrying out statistics on the occurrence frequency of garbage in the garden area and high-incidence areas, and generating a garbage inspection report in the garden area. The problems that a traditional manual inspection mode is low in efficiency, comprehensive coverage is difficult to achieve, inspection blind areas are likely to occur, and garbage cannot be found and treated in time are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of drone inspection, and in particular to a method and system for inspecting garbage in a garden area based on a drone. Background Art

[0002] With the acceleration of urbanization, urban gardens are expanding in size and their functions are becoming increasingly diverse. People's demands for the quality of garden environments are becoming increasingly stringent, and maintaining clean and beautiful gardens has become a key task in garden management. In recent years, drone technology has experienced rapid development. Its flexibility, ease of operation, and low-altitude flight capabilities allow it to quickly reach areas difficult to reach by human operators. Furthermore, the high-definition cameras and sensors onboard drones are constantly being upgraded, enabling them to capture high-quality images and data, providing technical support for garbage inspections in garden areas. Using drones for inspections can overcome the limitations of manual inspections and improve both efficiency and accuracy.

[0003] Traditional manual inspection methods are inefficient and difficult to achieve full coverage when dealing with large garden areas. They are prone to blind spots and are unable to detect and dispose of garbage in a timely manner. Therefore, more efficient inspection methods are needed to meet the needs of garden management. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention provides a method and system for garbage inspection in a garden area based on a drone, which can effectively solve the problems involved in the above-mentioned background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: In a first aspect, the present invention provides a method for inspecting garbage in a garden area based on a drone, comprising the following steps: acquiring environmental data in the garden area, constructing an environmental model in the garden area, and outputting environmental signals in the garden area; collecting tourist behavior patterns in various areas of the garden area and analyzing them to obtain tourist behavior signals in each area; acquiring obstacle feature data in the garden area and determining temporary no-fly zones for obstacles in the garden area; performing three-dimensional modeling of the garden area using a multi-view RGB-D camera carried by a drone to generate a garden area map; combining the environmental signals in the garden area, the tourist behavior signals in each area, and the temporary no-fly zones for obstacles in the garden area, and acquiring status feature data of each drone, planning a drone inspection path, and determining an optimal drone inspection path; performing a drone inspection of the garden area based on the optimal drone inspection path, transmitting and storing drone inspection images of the garden area to a data center, analyzing the transmission and storage process of the drone inspection images, and determining whether the transmission and storage of the drone inspection images are successful; analyzing the successfully transmitted and stored drone inspection images, detecting and annotating garbage in the garden area; and statistically analyzing the frequency and high-incidence areas of garbage in the garden area based on the garbage detection and annotation results in the garden area to generate a garbage inspection report for the garden area.

[0006] As a further method, environmental data within the garden area is obtained, an environmental model within the garden area is constructed, and an environmental signal within the garden area is output. The specific analysis process is as follows: environmental data within the garden area is obtained, including the ambient light intensity gz within the garden area, the ambient humidity sd within the garden area, and the maximum ambient wind speed fv within the garden area; based on the environmental data within the garden area, an environmental model within the garden area is constructed, and an environmental signal within the garden area is output;

[0007] The environmental model within the garden area, the specific analysis process is as follows:

[0008]

[0009] Where Yuq is the environmental signal in the garden area, and e is a natural constant.

[0010] As a further method, the behavior patterns of tourists in each area of ​​the garden area are collected and analyzed to obtain the behavior signals of tourists in each area. The specific analysis process is as follows: the behavior patterns of tourists in each area of ​​the garden area are collected, including the tourist density ρ in each area. people , the length of time tourists stay in each area St; based on the behavior patterns of tourists in each area of ​​the garden area, a comprehensive analysis is conducted to obtain the behavior signals of tourists in each area, which are used as the analysis basis for determining the optimal inspection path of the drone.

[0011] As a further method, obstacle feature data within the garden area is obtained to determine the temporary no-fly zone within the garden area. The specific analysis process is as follows: Obtain obstacle feature data within the garden area, including the straight-line distance Zl between the obstacle within the garden area and the garden control area, the vertical height Zh between the obstacle within the garden area and the ground, and the angle Zj between the obstacle within the garden area and the main gate of the garden control area; Based on the obstacle feature data within the garden area, a comprehensive analysis is performed to obtain the obstacle feature signal within the garden area. The specific analysis process is as follows:

[0012]

[0013] Where Jx is the characteristic signal of obstacles in the garden area, and e is a natural constant;

[0014] Obtain the mapping set of obstacle characteristic signals within the garden area and obstacle temporary no-fly zone division scheme within the garden area pre-stored in the database, determine the matching obstacle temporary no-fly zone division scheme within the garden area based on the obstacle characteristic signals within the current garden area; and determine the obstacle temporary no-fly zone within the garden area based on the obstacle temporary no-fly zone division scheme within the garden area.

[0015] As a further method, the garden area is three-dimensionally modeled using the multi-view RGB-D camera carried by the drone to generate a garden area map. The specific analysis process is as follows: the drone is equipped with a multi-view RGB-D camera array to synchronously collect color image data; each frame of RGB-D data is converted into a local point cloud using the TSDF truncated signed distance function algorithm, and the multi-view point clouds are aligned using the ICP iterative closest point algorithm, and the IMU / GPS data are integrated to eliminate the cumulative error; the lightweight MobileNetV3+PointNet model is used to semantically annotate vegetation, buildings, paths, trash cans and other elements in the point cloud; the RGB image is projected onto the point cloud surface to generate a textured triangular mesh model, and the holes are filled through Poisson surface reconstruction to output the garden area map.

[0016] As a further method, the environmental signals in the garden area, the behavioral signals of tourists in each area, and the temporary no-fly zones within the garden area are combined with the status characteristic data of each drone to plan the drone inspection path and determine the optimal inspection path for the drone. The specific analysis process is as follows: based on the garden area map, the terrain slope and vegetation density of the garden area are extracted, a topological relationship diagram is constructed, the dangerous areas in the garden area are identified, and the coordinates of the dangerous areas in the garden area are obtained; the status characteristic data of each drone are obtained, specifically including the remaining power Ds of each drone and the effective photosensitive area gs of each drone sensor; based on the status characteristic data of each drone, a comprehensive analysis is performed to obtain the status characteristic value of each drone. The specific analysis process is as follows:

[0017]

[0018] Where wf is the UAV state characteristic value, η is the UAV remaining power compensation coefficient stored in the database;

[0019] The terrain slope of the garden area, vegetation density, coordinates of the dangerous area in the garden area, characteristic values ​​of the status of each drone, environmental signals in the garden area, tourist behavior signals in each area, and temporary no-fly zones for obstacles in the garden area are stored as specified labels; the specified label-drone optimal inspection path mapping set pre-stored in the database is obtained, and based on the current specified label, the matching drone optimal inspection path is determined.

[0020] As a further method, the transmission and storage process of drone inspection images is analyzed to determine whether the transmission and storage of drone inspection images are successful. The specific analysis process is as follows: collecting drone inspection image transmission process data, including drone inspection image transmission bit error rate wm, drone inspection image transmission signal strength db, drone inspection image transmission signal-to-noise ratio SNR; analyzing the drone inspection image transmission process data to obtain the drone inspection image transmission factor;

[0021] The specific analysis process of the UAV inspection image transmission factor is as follows:

[0022]

[0023] Where Cs is the UAV inspection image transmission factor, α1 is the set weight factor of wm, α2 is the set weight factor of db, and α3 is the set weight factor of SNR;

[0024] Analyze the UAV inspection image transmission factor with the UAV inspection image definition transmission factor stored in the database; if the UAV inspection image transmission factor is lower than the UAV inspection image definition transmission factor, the UAV inspection image transmission is unsuccessful and needs to be retransmitted; if the UAV inspection image transmission factor is not lower than the UAV inspection image definition transmission factor, the UAV inspection image transmission is successful and the UAV inspection image is stored;

[0025] Collect drone inspection image storage process data, including the drone inspection image storage read and write speed dx v , the storage completeness rate of UAV inspection images wzl; analyze the storage process data of UAV inspection images to obtain the storage factor of UAV inspection images;

[0026] The specific analysis process of the UAV inspection image storage factor is as follows:

[0027] Cy=dx v *β1+wzl*β2;

[0028] Where Cy is the storage factor of the UAV inspection image, β1 is the set dx v The weight factor of β2 is the weight factor of wzl;

[0029] The drone inspection image storage factor is analyzed with the drone inspection image definition storage factor stored in the database; if the drone inspection image storage factor is lower than the drone inspection image definition storage factor, the drone inspection image storage is unsuccessful and needs to be retransmitted; if the drone inspection image storage factor is not lower than the drone inspection image definition storage factor, the drone inspection image storage is successful.

[0030] As a further method, the successfully transmitted and stored drone inspection images are analyzed to detect and mark garbage in the garden area. The specific analysis process is as follows: the pre-trained YOLOv8 model is used to infer the drone inspection images, identify garbage targets in the garden area and classify and mark scattered, piled, and hanging garbage to obtain the corresponding types of garbage in the garden area; semantic segmentation is performed to refine the detected garbage area at the pixel level and mark the boundaries; the pixel coordinates of the garbage in the drone inspection image are converted into the global coordinates of the garden map: based on the GPS positioning data, camera pitch angle, heading angle, and altitude during drone shooting, the latitude and longitude of the garbage are calculated, and the 2D detection results are mapped to the garden area map through projection, and the three-dimensional position of the garbage in the garden area is marked; according to the division of each area within the garden area, the garbage location in the garden area is associated with the area within the garden area to which it belongs.

[0031] As a further method, the frequency of garbage occurrence and high-incidence areas in the garden area are counted, and a garbage inspection report for the garden area is generated. The specific analysis process is: based on the three-dimensional position of garbage in the garden area, the coordinate density of garbage in the garden area is analyzed, and a three-dimensional density cloud is drawn to intuitively display the high-incidence areas of garbage in the garden area. The DBSCAN algorithm is used to cluster the three-dimensional positions of garbage in the garden area to identify high-incidence points of garbage accumulation in the garden area; the total number of garbage in the garden area, the corresponding types of garbage in the garden area, and the high-incidence points of garbage accumulation in the garden area are integrated into a garbage inspection report for the garden area.

[0032] The second aspect of the present invention provides a garbage inspection system in a garden area based on a drone, including an environmental signal output module in the area, a tourist behavior signal analysis module, a temporary no-fly zone determination module, a drone optimal inspection path determination module, a drone inspection image transmission, storage and analysis module, a garbage detection and labeling module and a garbage inspection report generation module, wherein: the environmental signal output module in the area is used to obtain environmental data in the garden area, build an environmental model in the garden area, and output environmental signals in the garden area; the tourist behavior signal analysis module is used to collect tourist behavior patterns in various areas in the garden area, and analyze to obtain tourist behavior signals in various areas; the temporary no-fly zone determination module is used to obtain obstacle feature data in the garden area and determine a temporary no-fly zone for obstacles in the garden area; the drone optimal inspection path determination module is used to perform three-dimensional construction of the garden area through the multi-view RGB-D camera carried by the drone The module generates a map of the garden area, combines the environmental signals in the garden area, the behavioral signals of tourists in each area and the temporary no-fly zones of obstacles in the garden area, obtains the status characteristic data of each drone, plans the drone inspection path, and determines the optimal drone inspection path; the drone inspection image transmission, storage and analysis module is used to conduct drone inspections in the garden area based on the optimal drone inspection path, transmit the drone inspection images of the garden area to the data center for storage, analyze the transmission and storage process of the drone inspection images, and determine whether the transmission and storage of the drone inspection images are successful; the garbage detection and labeling module is used to analyze the drone inspection images that have been successfully transmitted and stored, detect garbage in the garden area and label them; the garbage inspection report generation module is used to count the frequency of garbage occurrence and high-incidence areas in the garden area based on the garbage detection and labeling results in the garden area, and generate a garbage inspection report in the garden area.

[0033] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0034] (1) The present invention provides a method and system for inspecting garbage in a garden area based on a drone. By acquiring environmental data within the garden area to construct an environmental model and output environmental signals, and collecting and analyzing tourist behavior patterns to obtain behavioral signals, the present invention can fully understand the environmental conditions of the garden and the activities of tourists. This helps to more accurately grasp the overall characteristics of the garden area and provide rich background information for subsequent inspection work. By using a drone equipped with a multi-view RGB-D camera to perform three-dimensional modeling to generate a map of the garden area, and combining environmental signals and tourist behavior signals to plan inspection paths, the optimal inspection path can be determined, the blindness of the inspection can be avoided, the inspection efficiency can be improved, and the drone can be ensured to cover all areas of the garden in the shortest time.

[0035] (2) The present invention ensures data security and integrity by transmitting inspection images to a data center for storage and analyzing the transmission and storage process to determine whether it is successful. This allows subsequent image analysis to be performed based on accurate data, avoiding erroneous analysis due to data loss or damage. Analyzing images that have been successfully transmitted and stored can accurately detect and mark garbage within the garden area. This precise detection helps to promptly identify the location and amount of garbage, provide accurate guidance for subsequent cleanup work, and improve the pertinence and efficiency of garbage cleanup. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present invention is further described with reference to the accompanying drawings. However, the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative effort.

[0037] Figure 1 Schematic diagram of the method steps of the present invention.

[0038] Figure 2 This is a schematic diagram of system module connections of the present invention. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0040] Reference Figure 1 As shown, the first aspect of the present invention provides a garbage inspection method in a garden area based on a drone, including: acquiring environmental data in the garden area, constructing an environmental model in the garden area, and outputting environmental signals in the garden area.

[0041] The specific analysis process is as follows: obtaining environmental data within the garden area, including the ambient light intensity gz within the garden area, the ambient humidity sd within the garden area, and the maximum ambient wind speed fv within the garden area; based on the environmental data within the garden area, constructing an environmental model within the garden area, and outputting the environmental signal within the garden area.

[0042] It should be noted that, in this embodiment, all parameters are dimensionless as needed.

[0043] The environmental model within the garden area, the specific analysis process is as follows:

[0044]

[0045] Where Yuq is the environmental signal in the garden area, and e is a natural constant.

[0046] By collecting environmental data within the garden area, such as light intensity, humidity, and maximum wind speed, the garden environment is monitored from multiple dimensions. Previously, relying solely on manual inspections would have been difficult to comprehensively and accurately obtain this environmental information. This approach provides a more scientific and accurate understanding of the garden environment. For example, varying light and humidity conditions can affect the decomposition rate and distribution of waste.

[0047] Environmental signals within garden areas can be used as a reference for drone inspection route planning. For example, in high winds, drone flight stability may be affected. Based on these environmental signals, the inspection route or time can be adjusted appropriately to avoid inspection mission failures or inaccurate data collection due to harsh environments, thereby improving inspection efficiency and success rates.

[0048] Environmental factors have a certain impact on the presence and distribution of waste. Environmental signals derived from environmental models can be used as reference variables when analyzing waste detection results. For example, areas with high humidity may be more prone to organic waste accumulation. When analyzing the frequency and locations of waste occurrence, incorporating environmental signals can more accurately analyze the relationship between waste and the environment, providing a more scientific basis for decision-making in garden waste management.

[0049] The behavior patterns of tourists in each area of ​​the garden are collected and analyzed to obtain the behavior signals of tourists in each area.

[0050] The specific analysis process is: collecting tourist behavior patterns in each area of ​​the garden area, including the tourist density ρ in each area people , the length of time tourists stay in each area St; based on the behavior patterns of tourists in each area of ​​the garden area, a comprehensive analysis is conducted to obtain the behavior signals of tourists in each area, which are used as the analysis basis for determining the optimal inspection path of the drone.

[0051] Regional tourist behavior signals, the specific analysis process is as follows:

[0052]

[0053] Where Ac is the regional tourist behavior signal, and e is a natural constant.

[0054] By collecting visitor density information, we can identify areas within the garden that attract the most visitors. When planning drone inspection routes, we can avoid these densely populated areas, preventing disruption to visitors and even potential safety hazards, such as drone crashes and injuries. This also reduces the impact of factors like obstruction by visitors on the quality of garbage inspection imagery, ensuring smooth inspections.

[0055] Combined with visitor length of stay data, it's possible to analyze patterns of visitor activity in different areas. For areas with longer stays, inspections can be conducted during less frequented periods, ensuring effective inspections while minimizing the visitor experience. For example, in some garden performance areas, where visitors stay for extended periods and density is high, inspections can be scheduled between performances or after they conclude.

[0056] Incorporating regional visitor behavior signals into the optimal drone inspection route can make inspection route planning more scientific and rational. By comprehensively considering visitor behavior, drones can complete inspection missions along more efficient routes, reducing unnecessary flight distance and time, and improving inspection efficiency. For example, in areas with low visitor density and short stays, inspections can be accelerated or reduced in frequency, focusing resources on areas with high tourist activity and potentially high levels of waste.

[0057] Obtain obstacle feature data within the garden area and determine temporary no-fly zones within the garden area.

[0058] Obtain obstacle feature data within the garden area, including the straight-line distance Zl between the obstacle and the garden control area, the vertical height Zh between the obstacle and the ground, and the angle Zj between the obstacle and the main gate of the garden control area. Based on the obstacle feature data within the garden area, a comprehensive analysis is performed to obtain the obstacle feature signal within the garden area. The specific analysis process is as follows:

[0059]

[0060] Where Jx is the characteristic signal of obstacles in the garden area, and e is a natural constant;

[0061] Obtain the mapping set of obstacle characteristic signals within the garden area and obstacle temporary no-fly zone division scheme within the garden area pre-stored in the database, determine the matching obstacle temporary no-fly zone division scheme within the garden area based on the obstacle characteristic signals within the current garden area; and determine the obstacle temporary no-fly zone within the garden area based on the obstacle temporary no-fly zone division scheme within the garden area.

[0062] It can accurately define no-fly zones, preventing drones and other aircraft from colliding with obstacles (such as tall trees, buildings, and balloons), protecting aircraft equipment and preventing aircraft from falling and injuring people or damaging property. Determining no-fly zones based on scientific calculations provides a standardized management basis for garden management departments, making them more efficient and reasonable in handling aircraft-related matters.

[0063] The multi-view RGB-D camera carried by the drone is used to perform three-dimensional modeling of the garden area and generate a garden area map. The environmental signals within the garden area, the behavioral signals of tourists in each area, and the temporary no-fly zones within the garden area are combined with the status feature data of each drone to plan the drone inspection path and determine the optimal drone inspection path.

[0064] The specific analysis process is as follows: the drone is equipped with a multi-camera RGB-D camera array to synchronously collect color image data; each frame of RGB-D data is converted into a local point cloud using the TSDF truncated signed distance function algorithm, the multi-view point cloud is aligned using the ICP iterative closest point algorithm, and the IMU / GPS data is integrated to eliminate the cumulative error; the lightweight MobileNetV3+PointNet model is used to semantically annotate vegetation, buildings, paths, trash cans and other elements in the point cloud; the RGB image is projected onto the point cloud surface to generate a textured triangular mesh model, and the holes are filled through Poisson surface reconstruction to output a garden area map.

[0065] Based on the garden area map, the terrain slope and vegetation density of the garden area are extracted, a topological relationship diagram is constructed, the dangerous areas in the garden area are identified, and the coordinates of the dangerous areas in the garden area are obtained; the status characteristic data of each drone is obtained, including the remaining power Ds of each drone and the effective photosensitive area gs of each drone sensor; based on the status characteristic data of each drone, a comprehensive analysis is performed to obtain the status characteristic value of each drone. The specific analysis process is as follows:

[0066]

[0067] Where wf is the UAV state characteristic value, η is the UAV remaining power compensation coefficient stored in the database;

[0068] The terrain slope of the garden area, vegetation density, coordinates of the dangerous area in the garden area, characteristic values ​​of the status of each drone, environmental signals in the garden area, tourist behavior signals in each area, and temporary no-fly zones for obstacles in the garden area are stored as specified labels; the specified label-drone optimal inspection path mapping set pre-stored in the database is obtained, and based on the current specified label, the matching drone optimal inspection path is determined.

[0069] By collecting data using a multi-view RGB-D camera array mounted on a drone and processing it with algorithms like TSDF and ICP, a highly accurate three-dimensional map of the garden area can be generated. Compared to traditional two-dimensional maps or simple modeling methods, 3D maps can more realistically and comprehensively reflect the spatial distribution of garden terrain, vegetation, buildings, and other elements, providing a precise geographic information foundation for subsequent inspections.

[0070] Using a lightweight MobileNetV3+PointNet model for semantic annotation, the system can accurately distinguish between garden elements such as vegetation, buildings, paths, and trash cans. This helps to more clearly identify different scenes and objects during inspections. For example, it can accurately locate the location of a trash can, making it easier to determine the surrounding garbage situation and facilitating garbage detection and management.

[0071] By projecting the RGB image onto the point cloud surface to generate a textured triangular mesh model and using Poisson surface reconstruction to fill the holes, the model quality can be optimized, making the garden area map more complete and accurate, and improving data availability.

[0072] Inspection routes are planned based on multiple factors, including the slope of the garden area's terrain, vegetation density, coordinates of dangerous areas within the garden area, characteristic values ​​of each drone's status, environmental signals within the garden area, visitor behavior signals in each area, and temporary no-fly zones within the garden area. This fully considers the actual garden environment and various influencing factors, avoiding the limitations of single-factor planning, making the inspection route more tailored to actual needs and improving the feasibility and safety of inspections. By constructing a mapping set of designated labels and drone optimal inspection routes, the optimal inspection route is quickly matched based on the current label, achieving automation and efficiency in inspection route planning. This reduces the workload and subjectivity of manual planning, and can quickly determine the optimal inspection plan based on the characteristics of different garden areas, thereby improving inspection efficiency.

[0073] Based on the optimal drone inspection path, drone inspections are carried out in the garden area. The drone inspection images of the garden area are transmitted to the data center and stored. The transmission and storage process of the drone inspection images is analyzed to determine whether the transmission and storage of the drone inspection images are successful.

[0074] The specific analysis process is: collecting the data of the drone inspection image transmission process, including the drone inspection image transmission bit error rate wm, drone inspection image transmission signal strength db D , UAV inspection image transmission signal-to-noise ratio (SNR); analyze the UAV inspection image transmission process data to obtain the UAV inspection image transmission factor;

[0075] The specific analysis process of the UAV inspection image transmission factor is as follows:

[0076]

[0077] Where Cs is the UAV inspection image transmission factor, α1 is the set weight factor of wm, α2 is the set weight factor of db, and α3 is the set weight factor of SNR;

[0078] Analyze the UAV inspection image transmission factor with the UAV inspection image definition transmission factor stored in the database; if the UAV inspection image transmission factor is lower than the UAV inspection image definition transmission factor, the UAV inspection image transmission is unsuccessful and needs to be retransmitted; if the UAV inspection image transmission factor is not lower than the UAV inspection image definition transmission factor, the UAV inspection image transmission is successful and the UAV inspection image is stored;

[0079] Collect drone inspection image storage process data, including the drone inspection image storage read and write speed dx v , the storage completeness rate of UAV inspection images wzl; analyze the storage process data of UAV inspection images to obtain the storage factor of UAV inspection images;

[0080] The specific analysis process of the UAV inspection image storage factor is as follows:

[0081] Cy=dx v *β1+wzl*β2;

[0082] Where Cy is the storage factor of the UAV inspection image, β1 is the set dx v The weight factor of β2 is the weight factor of wzl;

[0083] The drone inspection image storage factor is analyzed with the drone inspection image definition storage factor stored in the database; if the drone inspection image storage factor is lower than the drone inspection image definition storage factor, the drone inspection image storage is unsuccessful and needs to be retransmitted; if the drone inspection image storage factor is not lower than the drone inspection image definition storage factor, the drone inspection image storage is successful.

[0084] By collecting data such as transmission bit error rate, signal strength, and signal-to-noise ratio, and using a formula to calculate the transmission factor of drone inspection images, a quantitative assessment of image transmission quality is achieved. Compared to relying solely on experience or simple judgment, this quantification method is more scientific and accurate, clearly reflecting the actual transmission process. Comparing the transmission factor with the defined transmission factor clearly determines whether the transmission was successful. If the transmission is unsuccessful, retransmission is performed, effectively preventing image data loss or errors due to transmission issues, ensuring transmission reliability, and ensuring that the data center receives complete and accurate inspection images, providing a reliable data foundation for subsequent analysis.

[0085] The storage factor is calculated by collecting data on storage read / write speed and storage integrity, comprehensively considering two key factors: speed and integrity. This comprehensive assessment of the storage process avoids the bias inherent in focusing solely on a single factor (such as read / write speed). By comparing the data with the defined storage factor to determine storage success and retransmitting if unsuccessful, potential storage issues, such as storage device failures and data write errors, can be promptly identified and resolved, ensuring that inspection images are effectively stored in the data center for subsequent access and analysis.

[0086] Timely detection and resolution of transmission and storage problems can avoid repeated inspections due to missing or incorrect data, saving time and resources and optimizing the efficiency of the entire garden waste inspection process.

[0087] The successfully transmitted and stored drone inspection images are analyzed to detect and mark garbage in the garden area.

[0088] The specific analysis process is as follows: use the pre-trained YOLOv8 model to infer the drone inspection images, identify garbage targets in the garden area and classify and mark scattered, piled, and hanging garbage to obtain the corresponding types of garbage in the garden area; perform semantic segmentation refinement, perform pixel-level segmentation on the detected garbage area, and mark the boundaries; convert the pixel coordinates of the garbage in the drone inspection image into the global coordinates of the garden map: based on the GPS positioning data, camera pitch angle, heading angle, and altitude during drone shooting, calculate the latitude and longitude of the garbage, map the 2D detection results to the garden area map through projection, and mark the three-dimensional position of the garbage in the garden area; according to the division of each area within the garden area, associate the garbage location in the garden area with the area within the garden area to which it belongs.

[0089] Using a pre-trained YOLOv8 model to reason about inspection images enables rapid and accurate identification of garbage within garden areas. YOLOv8's advanced object detection capabilities significantly improve the efficiency and accuracy of garbage detection compared to traditional manual recognition or simple algorithms, enabling timely detection of various types of garbage. Identified garbage is categorized as scattered, piled, or hanging, and pixel-level semantic segmentation and boundary annotation enable refined garbage classification and management. This refined processing helps garden management departments gain a clearer understanding of the specific form and distribution of garbage, providing a detailed basis for developing subsequent cleanup strategies, such as adopting different cleaning methods for different types of garbage.

[0090] The garbage pixel coordinates are converted to global coordinates on the garden map. Combined with the drone's GPS positioning data, camera attitude, and altitude, the longitude and latitude of the garbage are calculated and mapped onto the garden area map to indicate the three-dimensional location. This allows the garbage's spatial location within the garden to be accurately determined, making it easier for managers to quickly locate the garbage, improving cleaning efficiency and avoiding blind searches.

[0091] Based on the garden area division, garbage locations are associated with their respective areas, facilitating the classification, statistics, and management of garbage in different areas. Garden management departments can clearly understand the distribution and generation of garbage in each area, focus on and strengthen cleaning of high-incidence areas, and achieve refined and differentiated management of garden areas.

[0092] Based on the garbage detection and labeling results in the garden area, the frequency of garbage occurrence and high-incidence areas in the garden area are counted, and a garbage inspection report for the garden area is generated.

[0093] The specific analysis process is as follows: based on the three-dimensional position of garbage in the garden area, the coordinate density of garbage in the garden area is analyzed, a three-dimensional density cloud is drawn, and the high-incidence areas of garbage in the garden area are visually displayed. The DBSCAN algorithm is used to cluster the three-dimensional position of garbage in the garden area and identify the high-incidence points of garbage accumulation in the garden area; the total amount of garbage in the garden area, the corresponding types of garbage in the garden area, and the high-incidence points of garbage accumulation in the garden area are integrated into a garbage inspection report in the garden area.

[0094] By analyzing the coordinate density of garbage and creating a three-dimensional density cloud, we can clearly and intuitively display areas with high garbage incidence within the garden area. Combining the DBSCAN algorithm with three-dimensional garbage location clustering can accurately identify high-incidence garbage concentration points. This helps garden management departments precisely locate areas with serious garbage problems, replacing previous methods of relying on experience or rough judgments, and providing precise direction for subsequent resource allocation.

[0095] Counting the frequency of garbage occurrences provides quantitative data on garbage generation within the garden. Combined with information on high-incidence areas, this data comprehensively reflects the temporal and spatial distribution of garbage within the garden, providing basic data for in-depth analysis of the causes of garbage generation.

[0096] The garbage inspection report integrates information such as total garbage quantity, type, and hotspots, providing comprehensive and accurate decision-making for garden management departments. This report allows managers to rationally allocate cleaning staff, equipment, and other resources, developing more targeted cleaning and management strategies for high-incidence areas and different garbage types, thereby improving management efficiency and effectiveness.

[0097] Scientific statistical analysis and report generation reduce blindness and arbitrariness in management, avoid waste of resources, and achieve rational allocation of resources, thereby effectively improving the overall efficiency of garden waste management.

[0098] Reference Figure 2As shown, the second aspect of the present invention provides a garbage inspection system in a garden area based on a drone, including an environmental signal output module in the area, a visitor behavior signal analysis module, a temporary no-fly zone determination module, a drone optimal inspection path determination module, a drone inspection image transmission, storage and analysis module, a garbage detection and labeling module and a garbage inspection report generation module.

[0099] The regional environmental signal output module is used to obtain environmental data within the garden area, build an environmental model within the garden area, and output environmental signals within the garden area.

[0100] The tourist behavior signal analysis module is used to collect the tourist behavior patterns in each area of ​​the garden area and analyze the tourist behavior signals in each area.

[0101] The temporary no-fly zone determination module is used to obtain obstacle feature data within the garden area and determine the temporary no-fly zone for obstacles within the garden area.

[0102] The optimal inspection path determination module for drones is used to perform three-dimensional modeling of the garden area through the multi-view RGB-D camera carried by the drone, generate a garden area map, combine environmental signals within the garden area, tourist behavior signals in each area, and temporary no-fly zones for obstacles in the garden area, and obtain the status feature data of each drone to plan the drone inspection path and determine the optimal drone inspection path.

[0103] The UAV inspection image transmission, storage and analysis module is used to conduct UAV inspections in garden areas based on the optimal UAV inspection path, transmit the UAV inspection images of garden areas to the data center for storage, analyze the UAV inspection image transmission and storage process, and determine whether the UAV inspection images are successfully transmitted and stored.

[0104] The garbage detection and labeling module is used to analyze the successfully transmitted and stored drone inspection images, detect and label garbage in the garden area.

[0105] The garbage inspection report generation module is used to count the frequency of garbage occurrence and high-incidence areas in the garden area based on the garbage detection and labeling results in the garden area, and generate a garbage inspection report in the garden area.

[0106] The above content is merely an example and explanation of the structure of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the scope of protection of the present invention.

Claims

1. A method for inspecting garbage in a garden area based on a drone, characterized in that: The following steps are involved: Acquire environmental data within the garden area, build an environmental model within the garden area, and output environmental signals within the garden area; Collect the behavior patterns of tourists in each area of ​​the garden area and analyze them to obtain the behavior signals of tourists in each area; Obtain obstacle feature data within the garden area and determine temporary no-fly zones within the garden area; The multi-view RGB-D camera onboard the drone is used to perform 3D modeling of the garden area and generate a garden area map. This is combined with environmental signals within the garden area, visitor behavior signals in each area, and temporary no-fly zones within the garden area. The status feature data of each drone is then obtained to plan the drone inspection path and determine the optimal drone inspection path. Based on the optimal inspection path of drones, drone inspections are carried out in the garden area. The inspection images of the garden area are transmitted to the data center and stored. The transmission and storage process of the drone inspection images is analyzed to determine whether the transmission and storage of the drone inspection images are successful. Analyze the successfully transmitted and stored drone inspection images, detect and mark garbage in the garden area; Based on the garbage detection and labeling results in the garden area, the frequency of garbage occurrence and high-incidence areas in the garden area are counted, and a garbage inspection report for the garden area is generated.

2. The method for inspecting garbage in a garden area using a drone according to claim 1, characterized in that: The environmental data within the garden area is obtained, the environmental model within the garden area is constructed, and the environmental signal within the garden area is output. The specific analysis process is as follows: Obtain environmental data within the garden area, including the ambient light intensity gz within the garden area, the ambient humidity sd within the garden area, and the maximum ambient wind speed fv within the garden area; Based on the environmental data within the garden area, an environmental model within the garden area is constructed and the environmental signal within the garden area is output; The environmental model within the garden area, the specific analysis process is as follows: Where Yuq is the environmental signal in the garden area, and e is a natural constant.

3. The method for inspecting garbage in a garden area using a drone according to claim 1, characterized in that: The tourist behavior patterns of each area in the garden area are collected and analyzed to obtain tourist behavior signals of each area. The specific analysis process is as follows: Collect tourist behavior patterns in each area of ​​the garden area, including the tourist density ρ in each area people , length of stay of tourists in each area St; Based on the behavior patterns of tourists in each area of ​​the garden, a comprehensive analysis is conducted to obtain the behavior signals of tourists in each area, which are used as the analysis basis for determining the optimal inspection path of the drone.

4. The method for inspecting garbage in a garden area based on a drone according to claim 1, characterized in that: Obtaining obstacle feature data within the garden area and determining a temporary no-fly zone within the garden area involves the following specific analysis process: Obtain obstacle feature data within the garden area, specifically including the straight-line distance Zl between the obstacle and the garden control area, the vertical height Zh between the obstacle and the ground, and the angle Zj between the obstacle and the main gate of the garden control area. Based on the characteristic data of obstacles in the garden area, the characteristic signals of obstacles in the garden area are obtained through comprehensive analysis. The specific analysis process is as follows: Where Jx is the characteristic signal of obstacles in the garden area, and e is a natural constant; Obtaining a pre-stored mapping set of obstacle characteristic signals within the garden area and temporary no-fly zone division schemes for obstacles within the garden area in the database, and determining a matching temporary no-fly zone division scheme for obstacles within the garden area based on the obstacle characteristic signals within the current garden area; Based on the division scheme of temporary no-fly zones with obstacles in the garden area, temporary no-fly zones with obstacles in the garden area are determined.

5. The method for inspecting garbage in a garden area based on a drone according to claim 1, characterized in that: The multi-view RGB-D camera carried by the drone is used to perform three-dimensional modeling of the garden area and generate a garden area map. The specific analysis process is as follows: The drone is equipped with a multi-camera array of RGB-D cameras to simultaneously collect color image data; Each frame of RGB-D data is converted into a local point cloud using the TSDF truncated signed distance function algorithm. The multi-view point clouds are aligned using the ICP iterative closest point algorithm, and the IMU / GPS data are fused to eliminate the accumulated error. A lightweight MobileNetV3+PointNet model is used to semantically annotate elements such as vegetation, buildings, paths, and trash cans in point clouds. The RGB image is projected onto the point cloud surface to generate a textured triangular mesh model. The holes are filled through Poisson surface reconstruction to output a garden area map.

6. The method for inspecting garbage in a garden area based on a drone according to claim 5, characterized in that: The above analysis combines the environmental signals in the garden area, the behavior signals of tourists in each area, and the temporary no-fly zones within the garden area, and obtains the status characteristic data of each drone to plan the drone inspection path and determine the optimal drone inspection path. The specific analysis process is as follows: Based on the garden area map, the terrain slope and vegetation density of the garden area are extracted, a topological relationship diagram is constructed, the dangerous areas of the garden area are marked, and the coordinates of the dangerous areas of the garden area are obtained; Obtain the status characteristic data of each drone, including the remaining battery power Ds of each drone and the effective photosensitive area gs of each drone sensor; Based on the state characteristic data of each drone, a comprehensive analysis is performed to obtain the state characteristic value of each drone. The specific analysis process is as follows: Where wf is the UAV state characteristic value, η is the UAV remaining power compensation coefficient stored in the database; The terrain slope of the garden area, vegetation density, coordinates of dangerous areas in the garden area, characteristic values ​​of each drone status, environmental signals in the garden area, tourist behavior signals in each area, and temporary no-fly zones due to obstacles in the garden area are stored as designated labels; Obtain the pre-stored specified tag-drone optimal inspection path mapping set in the database, and determine the matching drone optimal inspection path based on the current specified tag.

7. The method for inspecting garbage in a garden area based on a drone according to claim 1, characterized in that: The UAV inspection image transmission and storage process is analyzed to determine whether the UAV inspection image is successfully transmitted and stored. The specific analysis process is as follows: Collect the data of the drone inspection image transmission process, including the drone inspection image transmission bit error rate wm, drone inspection image transmission signal strength db, and drone inspection image transmission signal-to-noise ratio SNR; The data of the UAV inspection image transmission process is analyzed to obtain the UAV inspection image transmission factor; The specific analysis process of the UAV inspection image transmission factor is as follows: Where Cs is the UAV inspection image transmission factor, α1 is the set weight factor of wm, α2 is the set weight factor of db, and α3 is the set weight factor of SNR; Analyze the transmission factor of the UAV inspection image with the defined transmission factor of the UAV inspection image stored in the database; If the UAV inspection image transmission factor is lower than the UAV inspection image defined transmission factor, the UAV inspection image transmission is unsuccessful and needs to be retransmitted; If the UAV inspection image transmission factor is not lower than the UAV inspection image defined transmission factor, the UAV inspection image transmission is successful and the UAV inspection image is stored; Collect drone inspection image storage process data, including the drone inspection image storage read and write speed dx v , UAV inspection image storage completeness rate wzl; Analyze the storage process data of UAV inspection images to obtain the storage factor of UAV inspection images; The specific analysis process of the UAV inspection image storage factor is as follows: Cy=dx v *β1+wzl*β2; Where Cy is the storage factor of the UAV inspection image, β1 is the set dx v The weight factor of β2 is the weight factor of wzl; Analyze the storage factor of the drone inspection image and the definition storage factor of the drone inspection image stored in the database; If the drone inspection image storage factor is lower than the drone inspection image defined storage factor, the drone inspection image storage fails and needs to be retransmitted; If the drone inspection image storage factor is not lower than the drone inspection image defined storage factor, the drone inspection image is stored successfully.

8. The method for inspecting garbage in a garden area based on a drone according to claim 1, characterized in that: The successfully transmitted and stored drone inspection images are analyzed to detect and mark garbage in the garden area. The specific analysis process is as follows: Use the pre-trained YOLOv8 model to reason about drone inspection images, identify garbage targets in the garden area, and classify them into scattered, piled, and hanging garbage, thereby obtaining the corresponding types of garbage in the garden area. Perform semantic segmentation refinement, perform pixel-level segmentation on the detected garbage areas, and mark the boundaries; Convert the pixel coordinates of the garbage in the drone inspection image to the global coordinates of the garden map: Based on the GPS positioning data, camera pitch angle, heading angle, and altitude during drone photography, the latitude and longitude of the garbage are calculated. The 2D detection results are projected onto a map of the garden area, marking the three-dimensional location of the garbage within the garden area. According to the division of each area in the garden area, the garbage location in the garden area is associated with the area within the garden area to which it belongs.

9. The method for inspecting garbage in a garden area based on a drone according to claim 1, characterized in that: The statistics of garbage occurrence frequency and high-incidence areas in the garden area are used to generate a garbage inspection report in the garden area. The specific analysis process is as follows: Based on the three-dimensional position of garbage in the garden area, the coordinate density of garbage in the garden area is analyzed, and a three-dimensional density cloud is drawn to intuitively display the high-incidence areas of garbage in the garden area. The DBSCAN algorithm is used to cluster the three-dimensional positions of garbage in the garden area and identify the high-incidence points of garbage aggregation in the garden area; The total amount of garbage in the garden area, the corresponding types of garbage in the garden area, and the high-incidence points of garbage accumulation in the garden area are integrated into a garbage inspection report in the garden area.

10. A garbage inspection system in a garden area based on a drone, applied to a garbage inspection method in a garden area based on a drone according to any one of claims 1 to 9, characterized in that: It includes an environmental signal output module within the region, a tourist behavior signal analysis module, a temporary no-fly zone determination module, an optimal drone inspection path determination module, a drone inspection image transmission, storage, and analysis module, a garbage detection and annotation module, and a garbage inspection report generation module, among which: The regional environmental signal output module is used to obtain environmental data within the garden area, build an environmental model within the garden area, and output environmental signals within the garden area; The tourist behavior signal analysis module is used to collect the tourist behavior patterns in each area of ​​the garden area and analyze the tourist behavior signals in each area; A temporary no-fly zone determination module is used to obtain characteristic data of obstacles in the garden area and determine a temporary no-fly zone for obstacles in the garden area; The optimal inspection path determination module for drones is used to perform three-dimensional modeling of the garden area using the multi-view RGB-D camera carried by the drone, generate a garden area map, and combine the environmental signals within the garden area, the behavior signals of tourists in each area, and the temporary no-fly zones within the garden area. It also obtains the status feature data of each drone to plan the drone inspection path and determine the optimal drone inspection path. The UAV inspection image transmission, storage and analysis module is used to conduct UAV inspections in garden areas based on the optimal UAV inspection path, transmit the UAV inspection images of garden areas to the data center for storage, analyze the UAV inspection image transmission and storage process, and determine whether the UAV inspection image transmission and storage are successful; The garbage detection and labeling module is used to analyze the successfully transmitted and stored drone inspection images, detect and label garbage in the garden area; The garbage inspection report generation module is used to count the frequency of garbage occurrence and high-incidence areas in the garden area based on the garbage detection and labeling results in the garden area, and generate a garbage inspection report in the garden area.

Images