Smart city environmental sanitation unmanned aerial vehicle real-time monitoring system based on multi-sensor fusion

Abstract

The invention relates to the technical field of smart city environmental sanitation monitoring, and discloses a smart city environmental sanitation unmanned aerial vehicle real-time monitoring system based on multi-sensor fusion, and the system comprises a sensor cooperation module which is used for constructing a sensing infrastructure and sorting sensor configuration information; the data fusion module is used for marking a fusion boundary interval and establishing a fusion group; the track planning module is used for setting and adjusting track points; the parameter calibration module is used for planning a flight path and calculating calibration parameters; the state tracking module is used for classifying the fusion groups and planning a flight path; and the result output module is used for outputting monitoring parameter information and a flight path plane expansion view. All the modules work cooperatively, multi-sensor data fusion, unmanned aerial vehicle track optimization, accurate parameter calibration and operation entity state tracking are achieved, environmental sanitation monitoring efficiency and accuracy are improved, and effective technical support is provided for smart city environmental sanitation management.

Description

Technical Field

[0001] The invention relates to the technical field of smart city environmental sanitation monitoring, and particularly to a real-time monitoring system for smart city environmental sanitation drones based on multi-sensor fusion. Background Technique

[0002] With the rapid advancement of the urbanization process, urban environmental sanitation management is facing unprecedented challenges. Traditional environmental sanitation operation models have problems such as low efficiency, difficult supervision, and high labor costs, and it is difficult to meet the needs of the development of smart cities.

[0003] In existing environmental sanitation monitoring technologies, the application of a single sensor has obvious limitations. For example, relying solely on cameras for monitoring is greatly affected by conditions such as weather and light, and the monitoring effect is not good in harsh environments; the coverage range of a single type of sensor is limited, and it is impossible to comprehensively and real-time obtain relevant information on urban environmental sanitation operations. At the same time, there is a lack of an effective coordination mechanism between different sensors, and data cannot be shared and fused, resulting in the overall performance of the monitoring system being difficult to improve.

[0004] The application of drones in environmental sanitation monitoring also faces many problems. Unreasonable flight path planning will lead to low flight efficiency of the drone and it cannot reach the designated area for monitoring in time; inaccurate parameter calibration will affect the accuracy and reliability of the monitoring data; untimely status tracking makes it difficult to effectively grasp the real-time status of environmental sanitation operation entities.

[0005] Existing environmental sanitation monitoring systems also have deficiencies in data processing. The data fusion boundary is not clear, and it is impossible to effectively integrate and analyze data from different sources and of different types; the data output form is single, which cannot meet diverse monitoring needs and is difficult to provide comprehensive and accurate support for environmental sanitation management decisions.

[0006] With the in-depth development of the concept of smart cities, higher requirements are put forward for the intelligence and informatization of urban environmental sanitation management. A monitoring system that can integrate multiple sensor resources, achieve efficient flight path planning of drones, accurate parameter calibration, and real-time status tracking is needed to improve the efficiency and level of urban environmental sanitation management. Summary of the Invention

[0007] The purpose of the present invention is to provide a real-time monitoring system for smart city environmental sanitation drones based on multi-sensor fusion to solve the problems raised in the above background technique.

[0008] To achieve the above purpose, the present invention provides the following technical solution: A real-time monitoring system for smart city environmental sanitation drones based on multi-sensor fusion, the system includes: Sensor coordination module: Build a drone perception infrastructure, and sort out a sensor configuration information list according to standard specifications; Data Fusion Module: Mark the data fusion boundary interval on the perception infrastructure, establish fusion groups, and associate one or more boundary intervals with the fusion groups; Track Planning Module: Set the initial track points of the UAV within the fusion group, adjust the positions of the track points, and edit the attribute data of the track points; Parameter Calibration Module: Plan the flight path of the UAV within the fusion group from the track position to the initial track point, and calculate the common calibration parameters of all track points in the fusion group to the initial track point; Status Tracking Module: Classify the fusion group, load and connect to the sanitation operation entity model, and plan the flight path from the initial track point to the connected entity; Result Output Module: Output the UAV monitoring parameter information and the planar expansion diagram of the flight path.

[0009] Preferably, in the Sensor Collaboration Module: The Sensor Collaboration Module includes an Infrastructure Derivation Unit and a Device Information Sorting Unit, where: The Infrastructure Derivation Unit derives a sensor collaboration infrastructure based on the UAV's original perception model. The sensor collaboration infrastructure contains all the information of the UAV's original perception model and an empty framework model for storing the path geometric information generated by subsequent monitoring; The Device Information Sorting Unit extracts and forms a sensor configuration information list from the sanitation monitoring technical standards. The attribute data includes device serial number, category code, category name, device deployment location, affiliated fusion group, and connected device serial number.

[0010] Preferably, in the Data Fusion Module: The Data Fusion Module includes an Interval Boundary Determination Unit and a Fusion Group Association Unit, where: The Interval Boundary Determination Unit finds the interval where the sensor is located in three-dimensional space according to the sensor configuration information list, and defines the interval or the created auxiliary interval as the boundary interval according to specific discrimination rules; the specific discrimination rules include signal sensitivity threshold discrimination, coverage range threshold discrimination, and device distribution density discrimination; the creation method of the auxiliary interval is a circular interval formed by expanding a preset width outward from the edge of the original interval, and the expansion direction is perpendicular to the normal of the original interval space; The Fusion Group Association Unit establishes the corresponding fusion group in three-dimensional space according to the fusion group information on the sensor configuration information list and associates it with the affiliated boundary interval. One fusion group can contain multiple boundary intervals, and one boundary interval belongs to only one fusion group.

[0011] Preferably, in the Track Planning Module: The Track Planning Module includes an Initial Point Automatic Setting Unit and a Position Data Management Unit, where: The initial point automatic setting unit supports automatically setting an initial track point at the center of the fusion grouping boundary interval. When the fusion grouping contains two boundary intervals, the initial track point is located on the boundary interval with a larger coverage range. When the fusion grouping contains multiple boundary intervals, the initial track point is located on the boundary interval closest to the center. The initial track point refers to the centralized positioning point of all connection entities of the UAV tracks in the fusion grouping. The position data management unit adjusts the position of the set initial track point and edits the data, including the number and identification of the initial track point.

[0012] Preferably, in the parameter calibration module: The parameter calibration module includes a track-group association unit and a calibration parameter calculation unit, where: The track-group association unit reads the sensor configuration information list and automatically associates it with the fusion grouping defined in the three-dimensional space according to the fusion grouping information where the UAV track is located. The calibration parameter calculation unit selects the UAV track that is the farthest from the initial track point at the edge of the fusion grouping as needed, plans the flight path from this track to the initial track point, and obtains the path length. When the distance of the UAV tracks in the fusion grouping from the interval edge is greater than the preset value, the track and calibration parameters are not actually planned. Automatically calculate the longest path length from each point of the boundary interval in the fusion grouping to the initial track point. The module uses the larger value of the two path lengths as the calibration parameter within the fusion grouping. If there is no former path length, directly use the latter path length as the calibration parameter within the fusion grouping.

[0013] Preferably, in the state tracking module: The state tracking module includes a fusion grouping assembly unit and a state tracking execution unit, where: The fusion grouping assembly unit establishes classification groups in the three-dimensional space according to the sensor configuration information list, and adds the fusion groupings to the corresponding classification groups. One classification group can contain multiple fusion groupings, one fusion grouping belongs to only one classification group, and the UAV tracks on all fusion groupings contained in one classification group are connected to the same connection entity. One classification group corresponds to one connection entity; load the connection entity model at the corresponding spatial position in the sensor collaboration infrastructure according to the actual position of the connection entity. The state tracking execution unit plans the flight path between the initial track point of the fusion grouping and the corresponding connection entity, marks the key nodes on the path as path control points in the three-dimensional space, and then sequentially connects the initial track point, each path control point, and the connection entity to generate a flight path model, and supports referencing existing flight path models to complete all path planning.

[0014] Preferably, in the result output module: The result output module includes a flight path expansion diagram output unit and a device statistical report output unit, where: The flight path expansion diagram output unit expands the flight path model into a two-dimensional expansion diagram for output, and marks the length data at each branch; The device statistical report output unit outputs a statistical report including device serial numbers, path lengths, locations, and connected device serial numbers.

[0015] Preferably, in the calibration parameter calculation unit: when the fusion grouping includes three or more boundary intervals, the initial track point position is determined by calculating the average coordinates of the geometric centers of the boundary intervals; the measurement method of the path length is the shortest continuous path along the surface contour of the UAV track, with the starting point being the center point of the UAV track deployment position and the ending point being the center point of the initial track point.

[0016] Preferably, in the state tracking execution unit: the setting rule of the path control point is to select a feature point on the flight path at every preset distance, and the feature points include turning points of intervals, device avoidance points, and spatial intersection points; the reference condition for the existing flight path model is that the included angle between the path direction and the current planned path does not exceed 30 degrees, and the path length difference is within the preset error range.

[0017] Preferably, in the device information sorting unit: the extraction basis of the sensor configuration information list includes the UAV size parameter table, the sensor performance specification, and the deployment interface specification; the storage format of the attribute data is a structured table, including columns of serial number, category code, category name, deployment location, fusion grouping, and connected device.

[0018] Compared with the prior art, the beneficial effects of the present invention are: Through the collaborative work of multiple modules, the efficient integration and collaborative perception of multi-sensor resources are realized, significantly improving the comprehensiveness and accuracy of environmental sanitation monitoring. The sensor collaboration module constructs a perfect perception infrastructure, and the sorted sensor configuration information list provides accurate basic data for subsequent data fusion and track planning. The data fusion module realizes the effective fusion of multi-source sensor data through scientific boundary interval determination and fusion grouping association, improving the utilization value of data and the overall performance of the monitoring system.

[0019] The track planning module can automatically set the initial track point according to the characteristics of the fusion grouping, and flexibly adjust the track point position and attribute data, optimizing the UAV flight path and improving the monitoring efficiency and coverage. The parameter calibration module ensures the accuracy of the UAV track and the precision of the monitoring data by reasonably planning the flight path and calculating the calibration parameters, providing a guarantee for the precise management of environmental sanitation operations.

[0020] The status tracking module classifies the fusion groups and loads the sanitation operation entity model, plans the flight path from the initial waypoint to the connected entity, realizes the real-time status tracking of the sanitation operation entity, and enables the management personnel to timely grasp the operation situation. The drone monitoring parameter information and the planar development diagram of the flight path output by the result output module provide intuitive and comprehensive data support for sanitation management decision-making, and improve the scientificity and effectiveness of management decision-making.

[0021] The system realizes the fusion and collaboration of multi-sensor data, improves the reliability and accuracy of the data, and provides more comprehensive information support for the sanitation management of smart cities. By optimizing the drone flight path planning, the flight time and energy consumption are reduced, the monitoring efficiency is improved, and the operation cost is reduced. Precise parameter calibration and real-time status tracking ensure the stability and reliability of the monitoring system, can timely detect and solve problems in sanitation operations, and improve the level and quality of urban environmental sanitation management. At the same time, the modular design of the system makes it have good scalability and compatibility, can adapt to the changing needs in the development process of smart cities, and lays a foundation for future function upgrades and technology expansion. Brief Description of the Drawings

[0022] Figure 1 is the working principle diagram of the real-time monitoring system of the smart city sanitation drone based on multi-sensor fusion described in the present invention; Figure 2 is the working flow chart of the data fusion module; Figure 3 is the working flow chart of the parameter calibration module; Figure 4 is the working flow chart of the status tracking module; Figure 5 is the flow chart for supplementing the execution of status tracking. Detailed Embodiments

[0023] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] Please refer to Figures 1-5 , for the real-time monitoring system of the smart city sanitation drone based on multi-sensor fusion involved in the present invention, the specific implementation steps are as follows: Build a sensor collaboration module, build the drone perception infrastructure, and sort out the sensor configuration information list according to the standard specifications. This module is used to build the foundation of drone perception and provide support for subsequent data collection and processing.

[0025] Establish a data fusion module, mark the data fusion boundary interval on the perception infrastructure, establish fusion groups, and associate one or more boundary intervals to the fusion groups. Through this module, the fusion processing of different sensor data is realized, improving the accuracy and reliability of the data.

[0026] Apply the flight path planning module to set the initial flight path points of the unmanned aerial vehicle (UAV) within the fusion group, adjust the positions of the flight path points, and edit the attribute data of the flight path points. This module plans a reasonable flight path for the UAV to ensure that the UAV can efficiently complete the monitoring task.

[0027] Utilize the parameter calibration module to plan the flight path position of the UAV within the fusion group and the flight path between the initial flight path points, and calculate the common calibration parameters of all flight path points in the fusion group to the initial flight path points. Through the calculation of the calibration parameters, the accuracy and stability of the UAV flight path are guaranteed.

[0028] With the help of the status tracking module, classify the fusion group, load and connect the sanitation operation entity model, and plan the flight path between the initial flight path point and the connected entity. This module is used to track the status and position of the UAV in real time to ensure that the UAV can fly according to the planned path.

[0029] With the help of the result output module, output the monitoring parameter information of the UAV and the plane expansion diagram of the flight path. Through this module, the monitoring results are presented to the user in an intuitive way, facilitating the user to view and analyze.

[0030] Embodiment 1: In the sensor collaboration module, the implementation method of this module is as follows: The sensor collaboration module includes two important components, namely the infrastructure derivation unit and the device information sorting unit. The two cooperate with each other to jointly realize the management and organization of sensor information.

[0031] For the infrastructure derivation unit, its work is to expand on the basis of the original perception model of the UAV. Specifically, the derived sensor collaboration infrastructure does not arise out of thin air but completely contains all the information in the original perception model of the UAV. The information in the original perception model covers various basic attributes and parameters of the UAV at the perception level, such as the type and quantity of sensors, the initial perception range, etc., which are all important bases for subsequent work. At the same time, in order to meet the storage requirements of path geometric information during subsequent monitoring, the infrastructure also specifically sets up an empty framework model. This empty framework model is like a container waiting to be filled. During subsequent monitoring operations of the UAV, various geometric information generated by its flight path, such as the coordinates of the path, the positions of turning points, the shape of the flight trajectory, etc., will be accurately stored in this empty framework model, providing an effective carrier for subsequent data processing and analysis.

[0032] The core task of the device information collation unit is to extract key information from the sanitation monitoring technical standards and form a list of sensor configuration information. During the extraction process, the information is not obtained randomly but based on clear criteria, including the drone size parameter table, sensor performance specifications, and deployment interface specifications, etc. The drone size parameter table details the physical size of the drone, which is of great significance for determining the installation location and spatial layout of the sensors on the drone, as different-sized drones may require different sensor installation methods. The sensor performance specifications comprehensively introduce various performance indicators of the sensors, such as detection accuracy, response time, measurement range, etc. These are the key to judging whether the sensors meet the sanitation monitoring requirements and are also important references for subsequent data fusion and processing. The deployment interface specifications stipulate the connection methods and communication protocols between the sensors and the drones as well as other devices, ensuring that the sensors can be properly connected to the system and data can be transmitted.

[0033] After extracting the relevant information, the device information collation unit will collate this information to form a list of sensor configuration information. The attribute data of this list contains multiple key items. First is the device serial number, which is like the unique identifier of each sensor, used to distinguish different sensor devices and facilitate management and invocation in the system. The category code and category name classify the sensors. For example, they can be divided into different categories such as image sensors, gas sensors, temperature sensors, etc. Through the category code and name, the type and function of the sensors can be quickly understood. The device deployment location specifies the specific installation location of the sensors on the drone or in the monitoring area, which is very important for subsequent data fusion and trajectory planning, as different deployment locations will affect the sensing range and data acquisition effect of the sensors. The affiliated fusion group information associates the sensors with subsequent data fusion processing and determines in which fusion group the sensor data will be processed. The connected device serial number records the connection relationship between this sensor and other devices, helping to build the device network structure of the entire system.

[0034] To facilitate the storage and management of this attribute data, the device information collation unit stores it in the form of a structured table. This table includes columns such as serial number column, category code column, category name column, deployment location column, fusion group column, and connected device column. Through this structured storage method, the list of sensor configuration information has a clear structure and good readability, and the system can easily query, modify, and invoke this data. For example, when it is necessary to view all the sensors within a certain fusion group, the relevant sensor information can be quickly filtered out through the fusion group column; when it is necessary to know the connected devices of a certain sensor, the relevant information can be obtained through the connected device column.

[0035] In the actual implementation process, the infrastructure derivation unit and the device information collation unit work collaboratively. The infrastructure derivation unit first constructs the sensor collaboration infrastructure, providing a framework and foundation for information storage for the device information collation unit. The device information collation unit then fills the specific sensor information into this infrastructure by extracting and collating the sensor configuration information, enabling the entire sensor collaboration module to comprehensively grasp the information of all sensors in the system. This collaborative working method ensures the integrity and accuracy of the sensor information, laying a solid foundation for the work of other modules such as the subsequent data fusion module and the trajectory planning module.

[0036] For example, after the sensor collaboration infrastructure is constructed, when a new sensor is added to the system, the device information collation unit will extract the information of this sensor from the relevant parameter tables, specifications, and manuals according to the environmental sanitation monitoring technical standards, form the corresponding attribute data, and fill it into the structured table. At the same time, the information of this sensor is associated with the empty frame model in the sensor collaboration infrastructure to store the path geometry information generated during its monitoring process. When the information of a certain sensor needs to be modified, the data in the structured table can also be updated through the device information collation unit to ensure the accuracy and timeliness of the sensor information in the system.

[0037] Through the above implementation methods of the infrastructure derivation unit and the device information collation unit in the sensor collaboration module, the effective management and organization of sensor information are realized, enabling the system to comprehensively and accurately grasp the configuration and status information of all sensors, providing strong support for the normal operation and efficient work of the entire intelligent city environmental sanitation UAV real-time monitoring system based on multi-sensor fusion. From the extraction, collation to storage of sensor information, each link is closely connected, forming a complete information management chain, ensuring the smooth transmission and effective utilization of sensor information in the system.

[0038] Embodiment 2: In the data fusion module, the implementation method of this module is as follows: The data fusion module includes an interval boundary determination unit and a fusion grouping association unit, which work collaboratively in three-dimensional space to achieve precise division and grouped management of the sensor data fusion area.

[0039] The core task of the interval boundary determination unit is to determine the effective range boundary of sensors in three-dimensional space based on the sensor configuration information list. Specifically, this unit first reads information such as the deployment location, type, and performance parameters of each sensor from the list. For example, the model of the sensor determines its signal transmission power and coverage angle, and the deployment location specifies its coordinate origin in space. Based on this information, the unit calculates the physical interval that the sensor can perceive in three-dimensional space. This interval usually takes the sensor as the center and forms a specific geometric shape, such as a hemisphere or a cylinder, according to its detection radius and angle.

[0040] After determining the original interval, the unit needs to further define the boundary interval according to specific discrimination rules. Among them, the signal sensitivity threshold discrimination means that when the sensor signal strength is lower than the preset threshold, this area is recognized as the boundary area because the sensor may not be able to accurately collect data at this time; the coverage range threshold discrimination takes the outermost end of the theoretical coverage range of the sensor as the boundary reference, and if it exceeds this range, there may be monitoring blind spots; the device distribution density discrimination is used in the scenario of multi-sensor overlapping deployment. When the number of sensors in the area is lower than a certain density, it is regarded as an area that needs to be separately demarcated.

[0041] When the boundary of the original interval cannot fully meet the data fusion requirements, the unit will create an auxiliary interval. The creation method of the auxiliary interval is to expand a preset width outward from the edge of the original interval to form an annular interval, and the expansion direction is perpendicular to the spatial normal of the original interval. For example, if the original interval is a cylinder distributed along the Z-axis direction and its spatial normal is the Z-axis direction, the auxiliary interval will expand outward along the Z-axis direction at the upper and lower bottom edges of the cylinder to form an annular structure that wraps the original interval. The value of the preset width is usually jointly determined by the signal attenuation characteristics of the sensor and the system accuracy requirements to ensure that the auxiliary interval can cover the effective edge area of the sensor signal.

[0042] The fusion grouping association unit establishes corresponding logical groupings in three-dimensional space based on the fusion grouping attributes in the sensor configuration information list. This unit first reads the fusion grouping identifier to which each sensor belongs in the list, such as "Group A", "Group B", etc., and then delimits the scope of action of each grouping in space according to the three-dimensional coordinate positions of the sensors. A fusion grouping can contain multiple boundary intervals. For example, when the monitoring areas of multiple sensors are adjacent or overlapping in space, their boundary intervals can be classified into the same fusion grouping to achieve centralized data processing.

[0043] During the association process, the unit will establish a mapping relationship between the boundary intervals and the fusion groups, ensuring that a boundary interval belongs to only one fusion group to avoid confusion in data processing. For example, if there are two adjacent boundary intervals formed by different sensors respectively, but their data needs to be fused according to the system plan, the unit will associate these two intervals with the same fusion group; if a boundary interval belongs to an independent monitoring area, it will be separately associated with a fusion group.

[0044] In specific implementation, the interval boundary determination unit and the fusion group association unit need to work alternately and iteratively. First, the interval boundary determination unit determines the preliminary boundary intervals based on the single-sensor data, and then the fusion group association unit aggregates or splits the intervals according to the grouping strategy. For example, when it is found that the sensor types in two adjacent boundary intervals are the same and the monitoring targets are consistent, the fusion group association unit will merge them into one fusion group and adjust the range of the boundary intervals; if there are too many boundary intervals in a fusion group resulting in reduced data processing efficiency, it may be split into multiple groups while redefining the boundaries of each interval.

[0045] During the 3D space modeling process, the unit will use the spatial coordinate system to accurately label each boundary interval and fusion group. For example, each boundary interval will be represented as a 3D cube containing the minimum and maximum coordinate values, or an irregular geometric body defined by multiple surface equations; the fusion group combines the geometric models of multiple boundary intervals into an overall model through set operations, facilitating subsequent calls by the track planning and parameter calibration modules.

[0046] In addition, this module also needs to handle the boundary definition problem in the overlapping area of multi-sensor data. When there is an overlap in the monitoring intervals of two sensors, the interval boundary determination unit will recalculate the boundary according to the signal strength superposition principle. For example, the intersection of the signal sensitivity thresholds of the two sensors is taken as the boundary of the overlapping area. At the same time, the fusion group association unit will assign the overlapping area to one of the fusion groups or determine the ownership according to attributes such as the department to which the sensor belongs and the monitoring priority.

[0047] During the operation of the system, the data fusion module will update the association relationship between the boundary intervals and the fusion groups in real time. For example, when a certain sensor fails or is redeployed, the interval boundary determination unit will recalculate its boundary interval, and the fusion group association unit will accordingly adjust the grouping ownership of this interval and synchronize the updated information to other modules to ensure that the perception model of the entire system is consistent with the actual deployment situation.

[0048] Through the above implementation manners, the data fusion module realizes the structured management of the sensor monitoring area in the three-dimensional space, converts the sensor data in the physical space into logically processable fusion groups, and provides a clear spatial division basis for the subsequent track planning module to determine the UAV flight area and the parameter calibration module to calculate path parameters, etc.

[0049] Embodiment 3: In the track planning module, the implementation manner of this module is specifically as follows: The track planning module consists of an initial point automatic setting unit and a position data management unit. The two cooperate with each other to complete the setting, adjustment and attribute management of the initial track points of the UAV within the fusion group, providing basic support for the flight path planning of the UAV.

[0050] The core function of the initial point automatic setting unit is to automatically determine the position of the initial track point within the boundary interval of the fusion group. This unit first obtains the number of boundary intervals included in the fusion group and the spatial parameters of each interval, such as the three-dimensional coordinate range and coverage area of each boundary interval. When the fusion group contains only one boundary interval, the unit directly sets the initial track point automatically at the geometric center position of this interval. The calculation of the geometric center is based on the extreme values of the three-dimensional coordinates of the interval. For example, for a cuboid-shaped boundary interval, its geometric center coordinates are .

[0051] When the fusion group contains two boundary intervals, the unit will compare the coverage ranges of the two intervals. The calculation of the coverage range needs to comprehensively consider the spatial dimensions of the interval. For example, for a cylindrical interval, the coverage range is determined by the bottom radius and height together; for an irregular interval, it is calculated through the three-dimensional volume formula. The initial track point will be set on the boundary interval with a larger coverage range and is also located at the geometric center position of this interval. If the coverage ranges of the two boundary intervals are the same, it can be determined according to the priority of sensor deployment or the spatial position of the interval. For example, select the interval closer to the center of the monitoring area.

[0052] When the fusion group contains multiple boundary intervals, the unit needs to determine the boundary interval closest to the center. Here, the "center" refers to the spatial centroid of all boundary intervals of the fusion group. The centroid coordinates are calculated by the weighted average of the geometric center coordinates and the coverage ranges of each interval. Specifically, the geometric center coordinates of each boundary interval are multiplied by its coverage range (or volume), and then all the results are added together. Finally, it is divided by the sum of the coverage ranges of all intervals to obtain the centroid coordinates of the fusion group. Then calculate the distance from the geometric center of each boundary interval to the centroid. The boundary interval with the smallest distance is the interval closest to the center, and the initial track point is set at the geometric center of this interval.

[0053] It should be clear that the initial track point is the centralized positioning point for all connected entities of the UAV track in the fusion group. The connected entities here include sanitation operation equipment, data transfer stations, and other objects that require UAV monitoring or data interaction. The setting of the initial track point provides a unified positioning reference for the connection of these entities, ensuring that the UAV can accurately dock with each entity during flight.

[0054] The position data management unit then performs subsequent position adjustment and data editing on the set initial track point. In practical applications, due to changes in environmental factors or monitoring requirements, the initial track point may need to be fine-tuned. For example, when new sensors or sanitation operation equipment are added to the fusion group, in order to enable the UAV to better cover the new monitoring targets, the initial track point needs to be moved a certain distance in the direction of the new targets. The operation of position adjustment supports manual input of coordinate values or dragging the track point through a three-dimensional spatial interaction interface. The system will display the adjusted position coordinates and the deviation value from the original position in real time.

[0055] For the attribute data editing of the initial track point, it mainly includes the modification of the number and identification. The number is used to uniquely identify each initial track point, facilitating management and invocation in the system. The numbering rule can be generated according to the fusion group number and the track point sequence. For example, "FG01-TP01" represents the first track point of fusion group 01. The identification is used to describe the function or characteristics of the track point, such as "starting point for garbage station monitoring", "starting point for main road inspection", etc. The identification information is in text form and can be edited by the operator according to actual needs.

[0056] When performing position adjustment and data editing, the position data management unit will interact with other modules. For example, after adjusting the position of the initial track point, the system will automatically synchronize the new position information to the parameter calibration module for recalculating calibration parameters; at the same time, the status tracking module will also obtain the updated position data to adjust the flight path planning from the initial track point to the connected entity.

[0057] In addition, the position data management unit also has a data verification function to ensure that the position of the edited initial track point meets the system requirements. For example, it checks whether the adjusted position is within the boundary interval of the fusion group. If it exceeds the interval range, a warning will be issued and the save will be rejected; it verifies whether the number is unique to avoid duplication with the numbers of other track points.

[0058] In the actual implementation process, the initial point automatic setting unit and the position data management unit work together. First, the initial point automatic setting unit sets the initial track point according to the boundary interval situation of the fusion grouping, and then the position data management unit makes necessary adjustments and attribute edits to it. For example, in a fusion grouping containing three boundary intervals, after the initial point automatic setting unit calculates the centroid position, it determines the boundary interval closest to the center and sets the initial track point. Subsequently, the operator discovers through the position data management unit that the track point is far from an important sanitation operation device, so moves it a certain distance in the direction of the device and modifies the identifier to "Monitoring starting point of device A".

[0059] In this way, the track planning module realizes the flexible management of the initial track points of the UAVs. It can not only automatically set reasonable initial positions according to spatial parameters, but also meet special monitoring requirements through manual intervention. The accurate setting and management of the initial track points lay a foundation for subsequent track planning, parameter calibration and other tasks, ensuring that the UAVs can efficiently complete the sanitation monitoring tasks according to the predetermined paths.

[0060] Embodiment 4: In the parameter calibration module, the implementation manner of this module is as follows: The parameter calibration module includes a track-group association unit and a calibration parameter calculation unit, which work together to calculate the UAV track calibration parameters within the fusion grouping to ensure the accuracy and stability of the UAV track.

[0061] The work of the track-group association unit is to read in the sensor configuration information list and automatically associate it with the fusion grouping defined in the three-dimensional space according to the fusion grouping information where the UAV track is located. For example, when there is a UAV track in the system and the sensor configuration information list corresponding to it shows that the affiliated fusion grouping is "Group A", the track-group association unit will search for the spatial range of the defined "Group A" in the three-dimensional space and then associate the UAV track with "Group A" to ensure that the subsequent calibration parameter calculation is carried out within the correct fusion grouping.

[0062] The work of the calibration parameter calculation unit is relatively complex and requires calculating the calibration parameters according to the UAV track situation within the fusion grouping. This unit will select the UAV track that is the most marginal and the farthest from the initial track point in the fusion grouping as needed. Here, "the most marginal" refers to the UAV track at the marginal position within the boundary interval of the fusion grouping, and "the farthest from the initial track point" is determined by calculating the distance from the center point of the track deployment position to the center point of the initial track point.

[0063] Suppose there are three UAV trajectories within a fusion group "Group B", namely Trajectory 1, Trajectory 2, and Trajectory 3. The initial trajectory points are set at the center of a certain boundary interval of this fusion group. Calculate the distances from these three trajectories to the initial trajectory points. It is found that the distance of Trajectory 3 is the farthest and it is at the edge position of the fusion group. Then the calibration parameter calculation unit will select Trajectory 3 for subsequent calibration parameter calculation.

[0064] The calibration parameter calculation unit will plan the flight path from this trajectory to the initial trajectory point and obtain the length of this path. The measurement method of the path length is the shortest continuous path along the surface contour of the UAV trajectory. The starting point of the path is the center point of the UAV trajectory deployment position, and the end point is the center point of the initial trajectory point. For example, if the coordinates of the center point of the deployment position of Trajectory 3 are (100, 200, 300) and the coordinates of the center point of the initial trajectory point are (500, 600, 700), then the planned flight path is the shortest continuous path from (100, 200, 300) to (500, 600, 700), and its length is calculated by the distance formula in three-dimensional space.

[0065] When the distances of the UAV trajectories in the fusion group from the interval edge are all greater than the preset value, the calibration parameter calculation unit does not actually plan the trajectories and calibration parameters. For example, in another fusion group "Group C", the distances of all UAV trajectories to the edges of their respective boundary intervals are all greater than the preset 50 meters. This indicates that these trajectories are within the safe range of the boundary interval and do not require additional calibration. Therefore, the calibration parameter calculation unit will not perform path planning and calibration parameter calculation on these trajectories.

[0066] The calibration parameter calculation unit will also automatically calculate the longest path length from each point in the boundary interval of the fusion group to the initial trajectory point. For example, the fusion group "Group D" contains two boundary intervals, Interval 1 and Interval 2. Calculate the distances from all points in Interval 1 to the initial trajectory point and find the longest path length L1; similarly, calculate the longest path length L2 from all points in Interval 2 to the initial trajectory point. Then the longest path length from each point in the boundary interval of this fusion group to the initial trajectory point is the larger value of L1 and L2.

[0067] The calibration parameter calculation unit uses the larger value of the two path lengths as the calibration parameter within this fusion group. These two path lengths are respectively the path length planned by selecting the UAV trajectory that is the farthest from the initial trajectory point at the most edge mentioned above, and the longest path length from each point in the boundary interval of the fusion group to the initial trajectory point. If there is no former path length, that is, when the distances of the UAV trajectories in the fusion group from the interval edge are all greater than the preset value and the former path length is not planned, then directly use the latter path length as the calibration parameter within this fusion group.

[0068] When the fusion group contains three or more boundary intervals, the initial track point position is determined by calculating the average coordinates of the geometric centers of each boundary interval. For example, the fusion group "Group E" contains three boundary intervals, namely Interval A, Interval B, and Interval C. The geometric center coordinates of each interval are (X1, Y1, Z1), (X2, Y2, Z2), and (X3, Y3, Z3) respectively. Then the position coordinates of the initial track point are the average of these three geometric center coordinates, that is .

[0069] In the actual implementation process, the track and group association unit and the calibration parameter calculation unit cooperate closely. First, the track and group association unit associates the UAV track with the correct fusion group, and then the calibration parameter calculation unit calculates the calibration parameters according to the situation of the fusion group. For example, for a fusion group "Group F" containing four boundary intervals, after the track and group association unit associates a certain UAV track with "Group F", the calibration parameter calculation unit first calculates the average coordinates of the geometric centers of the four boundary intervals to determine the initial track point position. Then, find the UAV track that is the most marginal and farthest from the initial track point in "Group F", plan its flight path to the initial track point, and obtain the path length L. At the same time, calculate the longest path length L' from each point of the boundary interval in "Group F" to the initial track point. Finally, compare L and L', and take the larger value as the calibration parameter of this fusion group.

[0070] Through the above implementation method of the parameter calibration module, the accurate calculation of the UAV track calibration parameters in the fusion group is realized, providing a reliable calibration basis for the flight of the UAV, ensuring that the UAV can fly accurately according to the predetermined track, and thus better completing the real-time monitoring task of smart city environmental sanitation.

[0071] Example 5: In the state tracking module, the implementation method of this module is as follows: The state tracking module consists of a fusion group assembly unit and a state tracking execution unit, which work together to achieve the classification management of the fusion group, the loading of the environmental sanitation operation entity model, and the planning of the flight path from the initial track point to the connection entity, so as to track the UAV state in real time and plan a reasonable flight path.

[0072] The fusion group assembly unit first establishes classification groups in the three-dimensional space according to the sensor configuration information list. For example, assume that there are multiple fusion groups in the system, which are used to monitor different types of environmental sanitation operation areas, such as main roads, garbage transfer stations, parks, etc. The fusion group assembly unit will establish corresponding classification groups according to these different operation types, such as "Main Road Monitoring Group", "Garbage Transfer Station Monitoring Group", "Park Monitoring Group", etc.

[0073] The fusion grouping assembly unit adds the fusion grouping to the corresponding classification group. A classification group can contain multiple fusion groupings, and one fusion grouping belongs to only one classification group. For example, the "main road monitoring group" may contain multiple fusion groupings responsible for monitoring different main roads, and each fusion grouping is uniquely assigned to the "main road monitoring group" classification group.

[0074] The UAV flight tracks on all the fusion groupings contained in a classification group are connected to the same connection entity, and a classification group corresponds to a connection entity one by one. Here, the connection entity can be a data processing center or a specific environmental sanitation operation management platform. For example, the "garbage transfer station monitoring group" classification group contains three fusion groupings, which monitor three different garbage transfer stations respectively. The UAV flight tracks on these three fusion groupings are all connected to the "garbage transfer station data processing center" connection entity to achieve centralized processing and management of data.

[0075] The fusion grouping assembly unit loads the connection entity model at the corresponding spatial position in the sensor collaboration infrastructure according to the actual position of the connection entity. For example, if the actual coordinate position of the "garbage transfer station data processing center" is (1000, 2000, 50), the fusion grouping assembly unit loads the three-dimensional model of this connection entity at the position with coordinates (1000, 2000, 50) in the sensor collaboration infrastructure, so that the position of the connection entity can be accurately found when planning the flight path later.

[0076] The status tracking execution unit is responsible for planning the flight path between the initial flight track point of the fusion grouping and the corresponding connection entity. For example, for a certain fusion grouping in the "garbage transfer station monitoring group", its initial flight track point is set at the center of the boundary interval of this fusion grouping, with coordinates (800, 1500, 100), and the coordinates of the corresponding connection entity "garbage transfer station data processing center" are (1000, 2000, 50). The status tracking execution unit needs to plan the flight path from (800, 1500, 100) to (1000, 2000, 50).

[0077] When planning the flight path, the status tracking execution unit will mark the key nodes on the path in the three-dimensional space as path control points. The setting rule of the path control points is to select a feature point at every preset distance on the flight path. The feature points include the turning points of the interval, the equipment avoidance points, and the spatial intersection points. Suppose the preset distance is 50 meters. In the flight path from the initial flight track point to the connection entity, the coordinates of the turning point of the interval encountered are (850, 1600, 90), the coordinates of the equipment avoidance point are (900, 1700, 80), and the coordinates of the spatial intersection point are (950, 1800, 70). Then these points will be marked as path control points.

[0078] The status tracking execution unit is sequentially connected to the initial waypoint, each path control point, and the connection entity to generate a flight path model. That is, it is connected in the order of (800, 1500, 100) → (850, 1600, 90) → (900, 1700, 80) → (950, 1800, 70) → (1000, 2000, 50) to generate a flight path model.

[0079] Meanwhile, the status tracking execution unit supports referencing an existing flight path model. The reference condition is that the angle between the path direction and the currently planned path does not exceed 30 degrees, and the difference in path length is within the preset error range. For example, there is already a flight path model in the system from the coordinates (700, 1400, 110) to (1000, 2000, 50), and the currently planned path is from (800, 1500, 100) to (1000, 2000, 50). The calculated angle between the directions of the two paths is 25 degrees, and the difference in path length is 30 meters, which is within the preset error range (assuming the preset error range is 50 meters). Then the status tracking execution unit can reference this existing flight path model, make appropriate adjustments and use it to improve the efficiency of path planning.

[0080] In another example, assume there is a classification group "Park Monitoring Group" that contains two fusion subgroups, which monitor the east and west regions of the park respectively. The fusion subgroup assembly unit adds these two fusion subgroups to the "Park Monitoring Group" classification group, connects their UAV trajectories to the connection entity "Park Management Platform", and then loads the model of the "Park Management Platform" in the sensor collaboration infrastructure. When the status tracking execution unit plans the flight path from the initial waypoints of these two fusion subgroups to the "Park Management Platform", path control points such as interval turning points and equipment avoidance points are marked on the path to generate a flight path model. The angle between the path direction from the initial waypoint of one fusion subgroup to the connection entity and the direction of an existing path in the system is 28 degrees, and the difference in path length is 40 meters, which meets the reference conditions. So this existing path model is referenced, reducing the workload of repeated planning.

[0081] Through the above implementation methods of the fusion subgroup assembly unit and the status tracking execution unit in the status tracking module, effective classification management of the fusion subgroups is achieved, as well as reasonable planning of the flight path from the initial waypoint to the connection entity. Thus, it is possible to track the status and position of the UAV in real time, ensure that the UAV flies according to the planned path, and complete the monitoring task of the sanitation operation area.

[0082] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variation thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0083] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A real-time monitoring system for urban sanitation drones based on multi-sensor fusion, characterized in that, include: Sensor collaboration module: Builds the drone perception infrastructure and compiles a sensor configuration information list based on standard specifications; Data fusion module: Marks data fusion boundary intervals on the perception infrastructure, establishes fusion groups, and associates one or more boundary intervals into the fusion group; Track planning module: set the initial track point of the drone within the fusion group, adjust the track point position and edit the track point attribute data; Parameter calibration module: plans the UAV track position within the fusion group and the flight path between the initial track point, and calculates the common calibration parameters from all track points in the fusion group to the initial track point; State tracking module: classifies fused groups, loads connected sanitation operation entity models, and plans flight paths from the initial track point to the connected entities; Result output module: outputs UAV monitoring parameter information and flight path plan expansion diagram.

2. The real-time monitoring system for smart city environmental sanitation drones based on multi-sensor fusion according to claim 1, wherein, In the sensor collaboration module: The sensor collaboration module includes an infrastructure derivation unit and a device information collation unit, where: The infrastructure derivation unit derives a sensor collaborative infrastructure based on the original UAV perception model. The sensor collaborative infrastructure contains all the information of the original UAV perception model and an empty frame model for storing the path geometry information generated by subsequent monitoring. The equipment information collation unit extracts and forms a list of sensor configuration information from the environmental sanitation monitoring technical standards. The attribute data includes the equipment serial number, category code, category name, equipment deployment location, fusion group, and connected equipment serial number.

3. The real-time monitoring system of the intelligent city environmental sanitation unmanned aerial vehicle based on multi-sensor fusion according to claim 1, characterized in that, In the data fusion module: The data fusion module includes an interval boundary determination unit and a fusion group association unit, wherein: The interval boundary determination unit locates the interval where the sensor is located in three-dimensional space based on the sensor configuration information list and defines this interval or a created auxiliary interval as a boundary interval based on specific discrimination rules. The specific discrimination rules include signal sensitivity threshold discrimination, coverage threshold discrimination, and device distribution density discrimination. The auxiliary interval is created by expanding the edge of the original interval outward to form a ring interval with a preset width, with the expansion direction perpendicular to the spatial normal of the original interval. The fusion group association unit establishes a corresponding fusion group in three-dimensional space according to the fusion group information on the sensor configuration information list, and associates it with the boundary interval. A fusion group can contain multiple boundary intervals, and a boundary interval belongs to only one fusion group.

4. The real-time monitoring system for urban sanitation drones based on multi-sensor fusion according to claim 1, wherein, In the trajectory planning module: The trajectory planning module includes an initial point automatic setting unit and a position data management unit, wherein: The automatic initial point setting unit supports automatically setting an initial track point at the center of the boundary interval of the fusion group. When the fusion group contains two boundary intervals, the initial track point is located on the boundary interval with the larger interval coverage. When the fusion group contains multiple boundary intervals, the initial track point is located on the boundary interval closest to the center. The initial track point refers to the centralized positioning point of all connected entities of the UAV track in the fusion group. The position data management unit adjusts the position of the set initial track point and edits the data, which includes the number and identification of the initial track point.

5. The real-time monitoring system for urban sanitation drones based on multi-sensor fusion according to claim 1, characterized in that, In the parameter calibration module: The parameter calibration module includes a track-group association unit and a calibration parameter calculation unit, where: The track-group association unit reads in the sensor configuration information list and automatically associates the UAV track with the fusion group defined in the three-dimensional space according to the fusion group information where the UAV track is located; The calibration parameter calculation unit selects the UAV track that is farthest from the initial track point at the edge of the fusion group as needed, plans the flight path from this track to the initial track point, and obtains the length of this path; if the distances between all UAV tracks in the fusion group and the boundary of the group space are greater than the preset threshold, no track planning and calibration parameter calculation are performed; automatically calculate the longest path lengths from each point in the boundary interval of the fusion group to the initial track point; the module takes the larger value of the two path lengths as the calibration parameter within this fusion group, and if there is no former path length, directly takes the latter path length as the calibration parameter within this fusion group.

6. The real-time monitoring system for urban sanitation drones based on multi-sensor fusion according to claim 1, characterized in that, In the state tracking module: The state tracking module includes a fusion group assembly unit and a state tracking execution unit, where: The fusion group assembly unit establishes classification groups in the three-dimensional space according to the sensor configuration information list, and adds the fusion groups to the corresponding classification groups. One classification group can contain multiple fusion groups, one fusion group belongs to only one classification group, and the UAV tracks on all fusion groups contained in one classification group are connected to the same connection entity. One classification group corresponds to one connection entity; load the connection entity model at the corresponding spatial position in the sensor collaboration infrastructure according to the actual position of the connection entity; The state tracking execution unit plans the flight path between the initial track point of the fusion group and the corresponding connection entity, marks the key nodes on the path as path control points in the three-dimensional space, and then connects the initial track point, each path control point and the connection entity in sequence to generate a flight path model, and supports the reference of existing flight path models to complete all path planning.

7. The real-time monitoring system for urban sanitation drones based on multi-sensor fusion according to claim 1, wherein, In the result output module: The result output module includes a path expansion diagram output unit and a device statistical report output unit, where: The path expansion diagram output unit expands the flight path model into a two-dimensional expansion diagram for output, and marks the length data at each branch; The device statistical report output unit outputs a statistical report including the device serial number, path length, location, and the serial number of the connected device.

8. The real-time monitoring system of the smart city environmental sanitation drone based on multi-sensor fusion according to claim 5, characterized in that, In the calibration parameter calculation unit: when the fusion group contains three or more boundary intervals, the position of the initial track point is determined by calculating the average coordinates of the geometric centers of each boundary interval; the measurement method of the path length is the shortest continuous path along the surface contour of the UAV track, with the starting point being the center point of the UAV track deployment position and the ending point being the center point of the initial track point.

9. The real-time monitoring system for urban sanitation drones based on multi-sensor fusion according to claim 6, characterized in that, In the state tracking execution unit: the setting rule of the path control points is to select a feature point at every preset distance on the flight path. The feature points include the turning points of the interval, the device avoidance points, and the spatial intersection points; the reference condition for the existing flight path model is that the included angle between the path direction and the currently planned path does not exceed 30 degrees, and the path length difference is within the preset error range.

10. The real-time monitoring system for urban sanitation drones based on multi-sensor fusion according to claim 2, characterized in that, In the device information sorting unit: The extraction basis of the sensor configuration information list includes the UAV size parameter table, the sensor performance specification, and the deployment interface specification; the storage format of the attribute data is a structured table, including columns for serial number, category code, category name, deployment location, fusion grouping, and connected device.

Images