Real-time monitoring system of smart city sanitation drones based on multi-sensor fusion

Abstract

The present invention relates to the field of smart city sanitation monitoring technology, and discloses a smart city sanitation drone real-time monitoring system based on multi-sensor fusion. The system includes a sensor collaboration module for building a perception infrastructure and sorting sensor configuration information; a data fusion module for marking fusion boundary intervals and establishing fusion groups; a track planning module for setting and adjusting track points; a parameter calibration module for planning flight paths and calculating calibration parameters; a state tracking module for classifying fusion groups and planning flight paths; and a result output module for outputting monitoring parameter information and a flight path planar expansion diagram. The modules work together to achieve multi-sensor data fusion, drone track optimization, precise parameter calibration, and operational entity status tracking, thereby improving the efficiency and accuracy of sanitation monitoring and providing effective technical support for smart city sanitation management.

Description

Technical Field

[0001] The present invention relates to the field of smart city sanitation monitoring technology, and specifically to a smart city sanitation drone real-time monitoring system based on multi-sensor fusion. Background Art

[0002] With the rapid advancement of urbanization, urban sanitation management faces unprecedented challenges. Traditional sanitation operations, characterized by low efficiency, difficult supervision, and high labor costs, struggle to meet the demands of smart city development.

[0003] Existing sanitation monitoring technologies suffer from significant limitations in the use of single sensors. For example, relying solely on cameras for monitoring is significantly affected by weather, lighting, and other conditions, resulting in poor monitoring effectiveness in harsh environments. A single type of sensor also has limited coverage, preventing comprehensive, real-time information on urban sanitation operations. Furthermore, the lack of effective coordination mechanisms between different sensors prevents data from being shared and integrated, hindering overall performance improvements in monitoring systems.

[0004] The use of drones in sanitation monitoring also faces numerous challenges. Irrational trajectory planning can lead to inefficient drone flight, preventing them from reaching designated areas for monitoring in a timely manner. Inaccurate parameter calibration can affect the accuracy and reliability of monitoring data. And inadequate status tracking can make it difficult to effectively monitor the real-time status of sanitation operations.

[0005] Existing sanitation monitoring systems also have shortcomings in data processing. The boundaries of data fusion are unclear, making it impossible to effectively integrate and analyze data from different sources and types. The single data output format cannot meet diverse monitoring needs, making it difficult to provide comprehensive and accurate support for sanitation management decisions.

[0006] The deepening development of the smart city concept has put forward higher requirements for the intelligent and information-based management of urban sanitation. A monitoring system that can integrate multiple sensor resources to achieve efficient drone trajectory planning, precise parameter calibration, and real-time status tracking is needed to improve the efficiency and level of urban sanitation management. Summary of the Invention

[0007] The purpose of the present invention is to provide a smart city sanitation drone real-time monitoring system based on multi-sensor fusion to solve the problems raised in the above background technology.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a real-time monitoring system for smart city sanitation drones based on multi-sensor fusion, the system comprising:

[0009] Sensor collaboration module: Builds the drone perception infrastructure and compiles a sensor configuration information list based on standard specifications;

[0010] Data fusion module: Marks data fusion boundary intervals on the perception infrastructure, establishes fusion groups, and associates one or more boundary intervals into the fusion group;

[0011] Track planning module: set the initial track point of the drone within the fusion group, adjust the track point position and edit the track point attribute data;

[0012] Parameter calibration module: plans the UAV track position within the fusion group and the flight path between the initial track point, and calculates the common calibration parameters from all track points in the fusion group to the initial track point;

[0013] State tracking module: classifies fused groups, loads connected sanitation operation entity models, and plans flight paths from the initial track point to the connected entities;

[0014] Result output module: outputs UAV monitoring parameter information and flight path plan expansion diagram.

[0015] Preferably, in the sensor collaboration module:

[0016] The sensor collaboration module includes an infrastructure derivation unit and a device information collation unit, where:

[0017] The infrastructure derivation unit derives a sensor collaborative infrastructure based on the original UAV perception model. The sensor collaborative infrastructure contains all the information of the original UAV perception model and an empty frame model for storing the path geometry information generated by subsequent monitoring.

[0018] The equipment information collation unit extracts and forms a list of sensor configuration information from the environmental sanitation monitoring technical standards. The attribute data includes the equipment serial number, category code, category name, equipment deployment location, fusion group, and connected equipment serial number.

[0019] Preferably, in the data fusion module:

[0020] The data fusion module includes an interval boundary determination unit and a fusion group association unit, wherein:

[0021] The interval boundary determination unit locates the interval where the sensor is located in three-dimensional space based on the sensor configuration information list and defines this interval or a created auxiliary interval as a boundary interval based on specific discrimination rules. The specific discrimination rules include signal sensitivity threshold discrimination, coverage threshold discrimination, and device distribution density discrimination. The auxiliary interval is created by expanding the edge of the original interval outward to form a ring interval with a preset width, with the expansion direction perpendicular to the spatial normal of the original interval.

[0022] The fusion group association unit establishes a corresponding fusion group in three-dimensional space according to the fusion group information on the sensor configuration information list, and associates it with the boundary interval. A fusion group can contain multiple boundary intervals, and a boundary interval belongs to only one fusion group.

[0023] Preferably, in the trajectory planning module:

[0024] The trajectory planning module includes an initial point automatic setting unit and a position data management unit, wherein:

[0025] The automatic initial point setting unit supports automatically setting an initial track point at the center of the boundary interval of the fusion group. When the fusion group contains two boundary intervals, the initial track point is located on the boundary interval with the larger interval coverage. When the fusion group contains multiple boundary intervals, the initial track point is located on the boundary interval closest to the center. The initial track point refers to the centralized positioning point of all connected entities of the UAV track in the fusion group.

[0026] The position data management unit adjusts the position of the set initial track point and edits the data, which includes the number and identification of the initial track point.

[0027] Preferably, in the parameter calibration module:

[0028] The parameter calibration module includes a track and group association unit and a calibration parameter calculation unit, wherein:

[0029] The track and group association unit reads the sensor configuration information list and automatically associates the UAV track with the fusion group defined in the three-dimensional space according to the fusion group information;

[0030] The calibration parameter calculation unit selects the UAV track at the edge of the fusion group that is farthest from the initial track point as needed, plans the flight path from the track to the initial track point, and obtains the path length; when the UAV tracks in the fusion group are all greater than the preset value from the interval edge, the track and calibration parameters are not actually planned; the longest path length from each point in the boundary interval of the fusion group to the initial track point is automatically calculated; the module uses the larger of the two path lengths as the calibration parameter within the fusion group, and if the former path length is not available, the latter path length is directly used as the calibration parameter within the fusion group.

[0031] Preferably, in the state tracking module:

[0032] The state tracking module includes a fusion group assembly unit and a state tracking execution unit, where:

[0033] The fusion group assembly unit establishes classification groups in three-dimensional space based on the sensor configuration information list and adds the fusion groups to the corresponding classification groups. A classification group can contain multiple fusion groups, and a fusion group belongs to only one classification group. The UAV tracks of all fusion groups contained in a classification group are connected to the same connection entity. There is a one-to-one correspondence between a classification group and a connection entity. The connection entity model is loaded into the corresponding spatial position in the sensor collaborative infrastructure according to the actual position of the connection entity.

[0034] The state tracking execution unit plans the flight path between the initial track points of the fusion group and the corresponding connection entities, marks the key nodes on the path as path control points in three-dimensional space, and then connects the initial track points, each path control point and the connection entity in sequence to generate a flight path model. It also supports referencing existing flight path models to complete all path planning.

[0035] Preferably, in the result output module:

[0036] The result output module includes a path expansion diagram output unit and an equipment statistical report output unit, where:

[0037] The path expansion diagram output unit expands the flight path model into a two-dimensional expansion diagram and outputs it, and marks the length data on each branch;

[0038] The device statistical report output unit outputs a statistical report including device serial number, path length, location, and connected device serial number.

[0039] Preferably, in the calibration parameter calculation unit: when the fusion group contains three or more boundary intervals, the position of the initial track point is determined by calculating the average coordinates of the geometric center of each boundary interval; the path length is measured as the shortest continuous path along the surface contour of the UAV track, the starting point of the path is the center point of the UAV track deployment position, and the end point is the center point of the initial track point.

[0040] Preferably, in the state tracking execution unit: the setting rule of the path control point is to select a feature point on the flight path at every preset distance, and the feature points include interval turning points, equipment avoidance points and space intersections; the reference condition of the existing flight path model is that the angle between the path direction and the current planned path does not exceed thirty degrees, and the path length difference is within a preset error range.

[0041] Preferably, in the device information collation unit: the basis for extracting the sensor configuration information list includes the drone size parameter table, the sensor performance manual and the deployment interface specification; the storage format of the attribute data is a structured table, including a serial number column, a category code column, a category name column, a deployment location column, a fusion group column and a connection device column.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The collaborative work of multiple modules achieves efficient integration and collaborative perception of multi-sensor resources, significantly improving the comprehensiveness and accuracy of sanitation monitoring. The sensor collaboration module builds a comprehensive perception infrastructure and compiles a list of sensor configuration information, providing accurate foundational data for subsequent data fusion and trajectory planning. The data fusion module effectively integrates multi-source sensor data through scientific boundary interval determination and fusion group association, enhancing the data's utility and the overall performance of the monitoring system.

[0044] The track planning module automatically sets initial track points based on the characteristics of fusion grouping and flexibly adjusts track point locations and attribute data, optimizing the drone's flight path and improving monitoring efficiency and coverage. The parameter calibration module ensures the accuracy of the drone's track and the precision of monitoring data by rationally planning flight paths and calculating calibration parameters, ensuring precise management of sanitation operations.

[0045] The status tracking module classifies the fused groups and loads the sanitation operation entity model. It then plans the flight path from the initial track point to the connected entity, enabling real-time status tracking of the sanitation operation entity, allowing managers to keep abreast of the operation status. The output module provides drone monitoring parameter information and a planar diagram of the flight path, providing intuitive and comprehensive data support for sanitation management decisions, improving the scientific nature and effectiveness of management decisions.

[0046] This system integrates and coordinates multi-sensor data, improving data reliability and accuracy and providing more comprehensive information support for smart city sanitation management. By optimizing drone trajectory planning, flight time and energy consumption are reduced, monitoring efficiency is improved, and operating costs are lowered. Precise parameter calibration and real-time status tracking ensure the stability and reliability of the monitoring system, enabling timely identification and resolution of sanitation operation issues, and improving the level and quality of urban sanitation management. Furthermore, the system's modular design ensures excellent scalability and compatibility, enabling it to adapt to the ever-changing needs of smart city development and laying the foundation for future functional upgrades and technological expansion. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a working principle diagram of the smart city sanitation drone real-time monitoring system based on multi-sensor fusion according to the present invention;

[0048] Figure 2 This is the workflow diagram of the data fusion module;

[0049] Figure 3 This is the workflow diagram of the parameter calibration module;

[0050] Figure 4 This is the workflow diagram of the status tracking module;

[0051] Figure 5 Flowchart that implements supplementary state tracking. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] See also Figure 1-Figure 5 The present invention relates to a smart city sanitation drone real-time monitoring system based on multi-sensor fusion, and the specific implementation steps are as follows:

[0054] Build a sensor collaboration module, establish a drone perception infrastructure, and compile a sensor configuration information list based on standard specifications. This module is used to build the foundation of drone perception and provide support for subsequent data collection and processing.

[0055] A data fusion module is established to mark data fusion boundary intervals on the perception infrastructure, create fusion groups, and associate one or more boundary intervals with the fusion group. This module enables the fusion of data from different sensors, improving data accuracy and reliability.

[0056] Use the Track Planning module to set the initial track points for the drone within the fusion group, adjust the track point positions, and edit the track point attribute data. This module plans a reasonable flight path for the drone, ensuring that the drone can complete the monitoring mission efficiently.

[0057] The parameter calibration module plans the flight paths of the drones within the fusion group and the initial track points. It also calculates the common calibration parameters from all track points within the fusion group to the initial track point. This calibration parameter calculation ensures the accuracy and stability of the drone's trajectory.

[0058] The state tracking module classifies the fused groups, loads the connected sanitation operation entity model, and plans the flight path from the initial track point to the connected entity. This module is used to track the drone's status and location in real time, ensuring that the drone flies according to the planned path.

[0059] The result output module outputs the drone monitoring parameter information and the flight path plan. This module presents the monitoring results to the user in an intuitive manner, making it easy for them to view and analyze them.

[0060] Example 1: In the sensor collaboration module, the implementation method of the module is as follows: the sensor collaboration module includes two important components: an infrastructure derivation unit and a device information collation unit, which cooperate with each other to jointly realize the management and organization of sensor information.

[0061] The infrastructure derivation unit's work is to expand upon the original drone perception model. Specifically, the derived sensor collaboration infrastructure is not created out of thin air, but rather fully encompasses all the information in the original drone perception model. The original perception model information here covers the various basic attributes and parameters of the drone at the perception level, such as the type and quantity of sensors, the initial perception range, etc., which are an important foundation for subsequent work. At the same time, in order to meet the storage requirements for path geometry information during subsequent monitoring, the infrastructure also specifically sets up an empty frame model. This empty frame model is like a container waiting to be filled. When the drone conducts subsequent monitoring operations, the various geometric information generated by its flight path, such as the coordinates of the path, the location of turning points, the shape of the flight trajectory, etc., will be accurately stored in this empty frame model, providing an effective carrier for subsequent data processing and analysis.

[0062] The core task of the equipment information collation unit is to extract key information from environmental sanitation monitoring technical standards and compile a list of sensor configuration information. This extraction process is not random, but based on clear criteria, including drone dimensional parameter tables, sensor performance specifications, and deployment interface specifications. The drone dimensional parameter table details the drone's physical dimensions, which is important for determining the sensor's installation location and spatial layout on the drone, as drones of different sizes may require different sensor installation methods. The sensor performance specification comprehensively describes the sensor's various performance indicators, such as detection accuracy, response time, and measurement range. This information is key to determining whether the sensor meets environmental sanitation monitoring requirements and is an important reference for subsequent data fusion and processing. The deployment interface specification specifies the connection method and communication protocol between the sensor and the drone and other equipment, ensuring that the sensor can normally access the system and transmit data.

[0063] After extracting the relevant information, the device information organization unit compiles this information into a sensor configuration list. This list's attribute data includes several key items. The first is the device serial number, which serves as a unique identifier for each sensor, distinguishing different sensor devices and facilitating management and access within the system. The category code and name categorize the sensor. For example, they can be categorized into image sensors, gas sensors, and temperature sensors. The category code and name provide a quick way to understand the sensor's type and function. The device deployment location specifies the sensor's specific installation location within the drone or surveillance area. This is crucial for subsequent data fusion and trajectory planning, as different deployment locations affect the sensor's sensing range and data collection performance. The fusion group information associates the sensor with the subsequent data fusion process, determining the fusion group in which the sensor data will be processed. The connected device serial number records the sensor's connection relationship with other devices, helping to establish the overall system's device network structure.

[0064] To facilitate the storage and management of this attribute data, the device information organization unit uses a structured table format. This table contains columns for serial number, category code, category name, deployment location, fusion group, and connected device. This structured storage method provides a clear structure and good readability for the sensor configuration information list, making it easy for the system to query, modify, and access this data. For example, when viewing all sensors within a fusion group, the fusion group column can be used to quickly filter out relevant sensor information; when determining the connected devices for a particular sensor, the connected device column can be used to obtain relevant information.

[0065] In actual implementation, the infrastructure derivation unit and the device information collation unit work together. The infrastructure derivation unit first builds the sensor collaboration infrastructure, providing a framework and foundation for information storage for the device information collation unit. The device information collation unit then extracts and organizes sensor configuration information, populating this infrastructure with specific sensor information. This enables the entire sensor collaboration module to fully understand the information of all sensors in the system. This collaborative approach ensures the integrity and accuracy of sensor information, laying a solid foundation for the subsequent work of other modules, such as the data fusion module and the trajectory planning module.

[0066] For example, after building the sensor collaborative infrastructure, when a new sensor is added to the system, the device information collation unit extracts the sensor's information from relevant parameter tables, manuals, and specifications based on environmental sanitation monitoring technical standards, generates corresponding attribute data, and enters it into a structured table. This data is then associated with the empty framework model in the sensor collaborative infrastructure to facilitate the subsequent storage of path geometry information generated during monitoring. If the information of a particular sensor needs to be modified, the device information collation unit can also update the data in the structured table, ensuring the accuracy and timeliness of the sensor information in the system.

[0067] The aforementioned implementation of the infrastructure derivation unit and device information organization unit within the sensor collaboration module enables effective management and organization of sensor information, enabling the system to comprehensively and accurately grasp the configuration and status of all sensors. This provides strong support for the normal operation and efficient operation of the entire multi-sensor fusion-based smart city sanitation drone real-time monitoring system. From sensor information extraction and organization to storage, each link is closely linked, forming a complete information management chain, ensuring the smooth transmission and effective utilization of sensor information within the system.

[0068] Example 2: In the data fusion module, the implementation method of the module is as follows: the data fusion module includes an interval boundary determination unit and a fusion group association unit, which work together in three-dimensional space to achieve accurate division and group management of sensor data fusion areas.

[0069] The core work of the interval boundary determination unit is to determine the effective range boundary of the sensor in three-dimensional space based on the sensor configuration information list. Specifically, the unit first reads information such as the deployment location, type, and performance parameters of each sensor from the list. For example, the sensor model determines its signal transmission power and coverage angle, and the deployment location clarifies its coordinate origin in space. Based on this information, the unit calculates the physical interval that the sensor can perceive in three-dimensional space. This interval is usually centered on the sensor and forms a specific geometric shape, such as a hemisphere or cylinder, based on its detection radius and angle.

[0070] After determining the original interval, the unit must further define the boundary interval based on specific judgment rules. Signal sensitivity threshold judgment means that when the sensor signal strength falls below a preset threshold, the area is identified as a boundary area, as the sensor may not be able to accurately collect data at this time. Coverage threshold judgment uses the farthest end of the sensor's theoretical coverage range as a boundary reference. If it exceeds this range, a monitoring blind spot may occur. Device distribution density judgment is used in scenarios where multiple sensors are deployed in an overlapping manner. When the number of sensors in an area falls below a certain density, it is considered an area that requires separate boundary demarcation.

[0071] When the boundary of the original interval cannot fully meet the data fusion requirements, the unit will create an auxiliary interval. The auxiliary interval is created by expanding the preset width outward from the edge of the original interval to form a ring interval, and the expansion direction is perpendicular to the spatial normal of the original interval. For example, if the original interval is a cylinder distributed along the Z axis, and its spatial normal is the Z axis, then the auxiliary interval will expand outward along the Z axis at the upper and lower bottom edges of the cylinder to form a ring structure that wraps the original interval. The value of the preset width is usually determined by the signal attenuation characteristics of the sensor and the system accuracy requirements to ensure that the auxiliary interval can cover the effective edge area of ​​the sensor signal.

[0072] The fusion group association unit establishes corresponding logical groups in three-dimensional space based on the fusion group attributes in the sensor configuration information list. This unit first reads the fusion group identifier of each sensor in the list, such as "Group A," "Group B," and then spatially defines the scope of each group based on the sensor's three-dimensional coordinate position. A fusion group can contain multiple boundary intervals. For example, when the monitoring areas of multiple sensors are spatially adjacent or overlapping, their boundary intervals can be assigned to the same fusion group to enable centralized data processing.

[0073] During the association process, the unit establishes a mapping relationship between boundary intervals and fusion groups, ensuring that a boundary interval belongs to only one fusion group, avoiding confusion in data processing. For example, if there are two adjacent boundary intervals formed by different sensors, but their data needs to be fused according to system planning, the unit will associate these two intervals with the same fusion group. If a boundary interval belongs to an independent monitoring area, it will be associated with a single fusion group.

[0074] In specific implementations, the interval boundary determination unit and the fusion group association unit work in alternating iterations. First, the interval boundary determination unit determines preliminary boundary intervals based on single-sensor data. Then, the fusion group association unit aggregates or splits the intervals according to the grouping strategy. For example, if two adjacent boundary intervals are found to have the same sensor type and the same monitoring target, the fusion group association unit will merge them into a single fusion group and adjust the range of the boundary intervals. If a fusion group has too many boundary intervals, resulting in reduced data processing efficiency, it may be split into multiple groups, and the boundaries of each interval will be redefined.

[0075] During the 3D modeling process, the unit uses a spatial coordinate system to precisely label each boundary interval and fusion group. For example, each boundary interval is represented as a 3D cube with minimum and maximum coordinate values, or as an irregular geometric shape defined by multiple surface equations. Fusion groups use set operations to combine the geometric models of multiple boundary intervals into a single, integrated model, making it easier to call into subsequent trajectory planning and parameter calibration modules.

[0076] This module also addresses the definition of boundaries for overlapping areas of multi-sensor data. When two sensors' monitoring intervals overlap, the interval boundary determination unit recalculates the boundaries based on the principle of signal strength superposition. For example, it uses the intersection of the two sensors' signal sensitivity thresholds as the boundary of the overlapping area. Furthermore, the fusion group association unit assigns the overlapping area to one of the fusion groups, or determines the assignment based on attributes such as the sensor's department and monitoring priority.

[0077] During system operation, the data fusion module updates the association between boundary intervals and fusion groups in real time. For example, if a sensor fails or is relocated, the interval boundary determination unit recalculates its boundary interval, and the fusion group association unit adjusts the grouping of that interval accordingly. This updated information is then synchronized with other modules to ensure that the entire system's perception model is consistent with the actual deployment.

[0078] Through the above implementation, the data fusion module realizes the structured management of the sensor monitoring area in the three-dimensional space, converts the sensor data in the physical space into logically processable fusion groups, and provides a clear spatial division basis for the subsequent trajectory planning module to determine the UAV flight area and the parameter calibration module to calculate the path parameters.

[0079] Example 3: In the trajectory planning module, the implementation method of this module is as follows: the trajectory planning module is composed of an initial point automatic setting unit and a position data management unit. The two cooperate with each other to complete the setting, adjustment and attribute management of the initial trajectory points of the drone within the fusion group, providing basic support for the flight path planning of the drone.

[0080] The core function of the initial point automatic setting unit is to automatically determine the position of the initial track point within the boundary interval of the fusion group. The unit first obtains the number of boundary intervals contained in the fusion group and the spatial parameters of each interval, such as the three-dimensional coordinate range and coverage area of ​​each boundary interval. When the fusion group contains only one boundary interval, the unit automatically sets the initial track point directly at the geometric center of the interval. The calculation of the geometric center is based on the three-dimensional coordinate extreme value of the interval. For example, for a boundary interval in the shape of a cuboid, its geometric center coordinates are .

[0081] When the fusion group contains two boundary intervals, the unit will compare the coverage of the two intervals. The calculation of the coverage range needs to take into account the spatial dimensions of the interval. For example, for a cylindrical interval, the coverage range is determined by the base radius and height; for an irregular interval, it is calculated using a three-dimensional volume formula. The initial track point will be set on the boundary interval with the larger coverage range and will also be located at the geometric center of the interval. If the coverage range of the two boundary intervals is the same, it can be determined based on the priority of sensor deployment or the spatial position of the interval, for example, selecting the interval closer to the center of the monitoring area.

[0082] When the fused group contains multiple boundary intervals, the unit needs to determine the boundary interval closest to the center. The "center" here refers to the spatial centroid of all boundary intervals of the fused group. The centroid coordinates are calculated by taking the weighted average of the geometric center coordinates of each interval and the coverage range. Specifically, the geometric center coordinates of each boundary interval are multiplied by its coverage range (or volume), and all the results are added together, and finally divided by the sum of the coverage ranges of all intervals to obtain the centroid coordinates of the fused group. The distance from the geometric center to the centroid of each boundary interval is then calculated. The boundary interval with the smallest distance is the interval closest to the center, and the initial track point is set at the geometric center of this interval.

[0083] It's important to note that the initial track point serves as the central location for all connected entities within the drone's trajectory within the fusion group. These connected entities include sanitation equipment, data transfer stations, and other objects that require drone monitoring or data exchange. The initial track point provides a unified positioning reference for these connections, ensuring the drone can accurately connect with each entity during flight.

[0084] The position data management unit performs subsequent position adjustments and data editing on the set initial track points. In actual applications, the initial track points may need to be fine-tuned due to environmental factors or changes in monitoring requirements. For example, when a new sensor or sanitation equipment is added to the fusion group, in order to enable the drone to better cover the new monitoring target, the initial track point needs to be moved a certain distance in the direction of the new target. The position adjustment operation supports manual input of coordinate values ​​or dragging the track point through the three-dimensional space interactive interface. The system will display the adjusted position coordinates and the deviation value from the original position in real time.

[0085] Editing the attribute data of initial track points primarily involves modifying their numbers and identifiers. Numbers uniquely identify each initial track point, facilitating management and access within the system. Numbering rules are generated based on the fusion group number and track point sequence. For example, "FG01-TP01" represents the first track point in fusion group 01. Identifiers describe the function or characteristics of a track point, such as "Garbage Station Monitoring Start Point" or "Main Road Inspection Start Point." Identifier information is in text format and can be edited by the operator based on actual needs.

[0086] The Position Data Management Unit interacts with other modules during position adjustments and data editing. For example, when adjusting the position of the initial track point, the system automatically synchronizes the new position information with the Parameter Calibration Module to recalculate the calibration parameters. Simultaneously, the Status Tracking Module also receives the updated position data and adjusts the flight path planning from the initial track point to the connected entity.

[0087] The position data management unit also includes data validation to ensure that the edited initial track point position meets system requirements. For example, it checks whether the adjusted position is within the boundary range of the fusion group. If it is outside the range, a warning will be issued and the save will be refused. It also verifies that the number is unique to avoid duplication with other track point numbers.

[0088] In actual implementation, the initial point automatic setting unit and the position data management unit work together. First, the initial point automatic setting unit sets the initial track point based on the boundary interval of the fusion group, and then the position data management unit makes necessary adjustments and attribute edits. For example, in a fusion group containing three boundary intervals, after the initial point automatic setting unit calculates the centroid position, it determines the boundary interval closest to the center and sets the initial track point. Subsequently, the operator discovers through the position data management unit that the track point is far away from an important sanitation operation equipment, so it is moved a certain distance toward the equipment and modified to "Equipment A Monitoring Start Point."

[0089] In this way, the trajectory planning module enables flexible management of the drone's initial track points, automatically setting a reasonable initial position based on spatial parameters while also enabling manual intervention to meet specific monitoring needs. Accurately setting and managing initial track points lays the foundation for subsequent track planning and parameter calibration, ensuring the drone can efficiently complete its sanitation monitoring mission along the predetermined path.

[0090] Example 4: In the parameter calibration module, the implementation method of the module is as follows: the parameter calibration module includes a track and group association unit and a calibration parameter calculation unit, which work together to realize the calculation of the calibration parameters of the drone track in the fusion group to ensure the accuracy and stability of the drone track.

[0091] The Track and Group Association Unit reads in the sensor configuration information list and automatically associates the drone track with the fusion group defined in 3D space based on the fusion group information. For example, if a drone track exists in the system and its corresponding sensor configuration information list shows that it belongs to "Group A", the Track and Group Association Unit will search the spatial range of the defined "Group A" in 3D space and then associate the drone track with "Group A", ensuring that subsequent calibration parameter calculations are performed within the correct fusion group.

[0092] The calibration parameter calculation unit is relatively complex and requires calculating calibration parameters based on the UAV tracks within the fusion group. This unit will select the UAV track at the edge of the fusion group that is farthest from the initial track point. "Frontier" here refers to the UAV track at the edge of the boundary interval of the fusion group, while "farthest from the initial track point" is determined by calculating the distance from the center point of the track's deployment location to the center point of the initial track point.

[0093] Assume that a fused group, "Group B," contains three drone tracks: Track 1, Track 2, and Track 3. The initial track point is set at the center of a boundary interval within the fused group. When calculating the distances from these three tracks to the initial track point, Track 3 is found to be the farthest away and located at the edge of the fused group. Therefore, the calibration parameter calculation unit selects Track 3 for subsequent calibration parameter calculations.

[0094] The calibration parameter calculation unit plans the flight path from the track to the initial track point and calculates the path length. Path length is measured as the shortest continuous path along the drone's track surface contour, starting at the center of the drone's track deployment location and ending at the center of the initial track point. For example, if the coordinates of the deployment location center of Track 3 are (100, 200, 300) and the coordinates of the initial track point center are (500, 600, 700), then the planned flight path is the shortest continuous path from (100, 200, 300) to (500, 600, 700), and its length is calculated using the distance formula in three-dimensional space.

[0095] If all drone tracks in a fused group are further away from the edge of the interval than a preset value, the calibration parameter calculation unit does not actually plan tracks or calibrate parameters. For example, in another fused group, "Group C," all drone tracks are further away from the edge of their respective boundary intervals than the preset 50 meters. This indicates that these tracks are within the safe range of the boundary interval and do not require additional calibration. Therefore, the calibration parameter calculation unit does not perform path planning or calibration parameter calculation for these tracks.

[0096] The calibration parameter calculation unit also automatically calculates the longest path length from each point in the boundary intervals of the fused group to the initial track point. For example, fused group "Group D" contains two boundary intervals, Interval 1 and Interval 2. The distances from all points in Interval 1 to the initial track point are calculated to find the longest path length, L1. Similarly, the longest path length, L2, from all points in Interval 2 to the initial track point is calculated. The longest path length from each point in the boundary intervals of the fused group to the initial track point is then the larger of L1 and L2.

[0097] The calibration parameter calculation unit uses the larger of the two path lengths as the calibration parameter within the fused group. These two path lengths are the path length of the previously mentioned UAV track planning that is farthest from the initial track point, and the longest path length from each point in the boundary interval of the fused group to the initial track point. If the former path length is not available, that is, when the UAV tracks in the fused group are all greater than the preset value from the interval edge, the former path length is not planned, then the latter path length is directly used as the calibration parameter within the fused group.

[0098] When the fusion group contains three or more boundary intervals, the initial track point position is determined by calculating the average coordinates of the geometric centers of each boundary interval. For example, the fusion group "Group E" contains three boundary intervals, interval A, interval B, and interval C. The geometric center coordinates of each interval are (X1, Y1, Z1), (X2, Y2, Z2), and (X3, Y3, Z3). Then the position coordinates of the initial track point are the average of the three geometric center coordinates, that is, .

[0099] During the actual implementation process, the track and group association unit and the calibration parameter calculation unit work closely together. First, the track and group association unit associates the drone track with the correct fusion group, and then the calibration parameter calculation unit calculates the calibration parameters based on the situation of the fusion group. For example, for a fusion group "Group F" containing four boundary intervals, after the track and group association unit associates a drone track with "Group F", the calibration parameter calculation unit first calculates the average coordinates of the geometric centers of the four boundary intervals to determine the position of the initial track point. Next, find the drone track at the edge of "Group F" that is farthest from the initial track point, plan its flight path to the initial track point, and obtain the path length L. At the same time, calculate the longest path length L' from each point in the boundary interval of "Group F" to the initial track point. Finally, compare L and L', and take the larger value as the calibration parameter of the fusion group.

[0100] Through the above implementation of the parameter calibration module, accurate calculation of the calibration parameters of the drone track within the fusion group is achieved, providing a reliable calibration basis for the flight of the drone, ensuring that the drone can fly accurately according to the predetermined track, thereby better completing the real-time monitoring task of smart city sanitation.

[0101] Example 5: In the state tracking module, the implementation method of this module is as follows: the state tracking module is composed of a fusion group assembly unit and a state tracking execution unit. The two work together to realize the classification management of the fusion group, the loading of the sanitation operation entity model, and the planning of the flight path from the initial track point to the connection entity, thereby tracking the drone status in real time and planning a reasonable flight path.

[0102] The fusion grouping assembly unit first creates classification groups in three-dimensional space based on the sensor configuration information list. For example, suppose the system has multiple fusion groups for monitoring different types of sanitation operation areas, such as main roads, waste transfer stations, and parks. The fusion grouping assembly unit will create corresponding classification groups based on these different operation types, such as "main road monitoring group," "waste transfer station monitoring group," and "park monitoring group."

[0103] The fusion group assembly unit adds the fusion group to the corresponding classification group. A classification group can contain multiple fusion groups, and a fusion group belongs to only one classification group. For example, the "main road monitoring group" may contain multiple fusion groups responsible for monitoring different main roads. Each fusion group uniquely belongs to the "main road monitoring group" classification group.

[0104] All drone tracks from all fusion groups within a classification group are connected to the same connection entity, with a one-to-one correspondence between one classification group and one connection entity. This connection entity can be a data processing center or a specific sanitation operation management platform. For example, the "Waste Transfer Station Monitoring Group" classification group contains three fusion groups, each monitoring three different waste transfer stations. The drone tracks from these three fusion groups are all connected to the "Waste Transfer Station Data Processing Center" connection entity, enabling centralized data processing and management.

[0105] The fusion grouping and assembly unit loads the connected entity model into the sensor collaborative infrastructure at the corresponding spatial location based on the connected entity's actual location. For example, if the actual coordinates of the "Garbage Transfer Station Data Processing Center" are known to be (1000, 2000, 50), the fusion grouping and assembly unit loads the 3D model of the connected entity at the coordinates (1000, 2000, 50) in the sensor collaborative infrastructure, ensuring accurate location of the connected entity during subsequent flight path planning.

[0106] The state tracking execution unit is responsible for planning the flight path from the initial track point of the fused group to the corresponding connected entity. For example, for a fused group in the "Waste Transfer Station Monitoring Group," its initial track point is set at the center of the fused group's boundary interval, with coordinates (800, 1500, 100). The corresponding connected entity, the "Waste Transfer Station Data Processing Center," has coordinates (1000, 2000, 50). The state tracking execution unit needs to plan a flight path from (800, 1500, 100) to (1000, 2000, 50).

[0107] When planning a flight path, the state tracking execution unit marks key nodes on the path in three-dimensional space as path control points. The rule for setting path control points is to select a feature point on the flight path at every preset distance. Feature points include interval turning points, device avoidance points, and space intersections. Assuming the preset distance is 50 meters, in the flight path from the initial track point to the connecting entity, the coordinates of the interval turning point are (850, 1600, 90), the coordinates of the device avoidance point are (900, 1700, 80), and the coordinates of the space intersection are (950, 1800, 70), then these points will be marked as path control points.

[0108] The state tracking execution unit sequentially connects the initial track point, each path control point, and the connection entity to generate a flight path model. Specifically, the flight path model is generated in the order (800, 1500, 100) → (850, 1600, 90) → (900, 1700, 80) → (950, 1800, 70) → (1000, 2000, 50).

[0109] The state tracking execution unit also supports referencing existing flight path models. The reference conditions require that the angle between the path and the currently planned path does not exceed 30 degrees, and the path length difference is within a preset error range. For example, if the system already has a flight path model from coordinates (700, 1400, 110) to (1000, 2000, 50), and the currently planned path is from (800, 1500, 100) to (1000, 2000, 50), the calculated angle between the two paths is 25 degrees, and the path length difference is 30 meters, which is within the preset error range (assuming the preset error range is 50 meters). In this case, the state tracking execution unit can reference the existing flight path model and make appropriate adjustments to it before using it to improve the efficiency of path planning.

[0110] In another example, suppose there is a classification group called "Park Monitoring Group" that contains two fusion groups, one monitoring the east and west areas of the park. The fusion group assembly unit adds these two fusion groups to the "Park Monitoring Group" classification group and connects their drone tracks to the "Park Management Platform" connection entity. The model of the "Park Management Platform" is then loaded into the sensor collaborative infrastructure. When the state tracking execution unit plans the flight paths from the initial track points of the two fusion groups to the "Park Management Platform", path control points such as section turning points and equipment avoidance points are marked on the paths to generate a flight path model. The path direction from the initial track point of one of the fusion groups to the connection entity is at an angle of 28 degrees to the direction of an existing path in the system, and the path length difference is 40 meters, which meets the reference conditions. Therefore, the existing path model is referenced, reducing the workload of repeated planning.

[0111] Through the above-mentioned implementation methods of the fusion group assembly unit and the state tracking execution unit in the state tracking module, effective classification management of the fusion group and reasonable planning of the flight path from the initial track point to the connection entity are achieved, so that the status and position of the drone can be tracked in real time, ensuring that the drone flies according to the planned path and completing the monitoring task of the sanitation operation area.

[0112] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0113] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A smart city sanitation drone real-time monitoring system based on multi-sensor fusion, characterized by: include: Sensor collaboration module: Builds the drone perception infrastructure and compiles a sensor configuration information list based on standard specifications; Data fusion module: Marks data fusion boundary intervals on the perception infrastructure, establishes fusion groups, and associates one or more boundary intervals into the fusion group; Track planning module: set the initial track point of the drone within the fusion group, adjust the track point position and edit the track point attribute data; Parameter calibration module: plans the UAV track position within the fusion group and the flight path between the initial track point, and calculates the common calibration parameters from all track points in the fusion group to the initial track point; State tracking module: classifies fused groups, loads connected sanitation operation entity models, and plans flight paths from the initial track point to the connected entities; Result output module: outputs UAV monitoring parameter information and flight path plan expansion diagram; In the parameter calibration module: The parameter calibration module includes a track and group association unit and a calibration parameter calculation unit, wherein: The track and group association unit reads the sensor configuration information list and automatically associates the UAV track with the fusion group defined in the three-dimensional space according to the fusion group information; The calibration parameter calculation unit selects the UAV track at the edge of the fusion group that is farthest from the initial track point as needed, plans the flight path from the track to the initial track point, and obtains the path length; when the UAV tracks in the fusion group are all greater than the preset value from the interval edge, the track and calibration parameters are not actually planned; the longest path length from each point in the boundary interval of the fusion group to the initial track point is automatically calculated; the module uses the larger of the two path lengths as the calibration parameter within the fusion group, and if the former path length is not available, the latter path length is directly used as the calibration parameter within the fusion group.

2. The smart city sanitation drone real-time monitoring system based on multi-sensor fusion according to claim 1 is characterized in that: In the sensor collaboration module: The sensor collaboration module includes an infrastructure derivation unit and a device information collation unit, where: The infrastructure derivation unit derives a sensor collaborative infrastructure based on the original UAV perception model. The sensor collaborative infrastructure contains all the information of the original UAV perception model and an empty frame model for storing the path geometry information generated by subsequent monitoring. The equipment information collation unit extracts and forms a list of sensor configuration information from the environmental sanitation monitoring technical standards. The attribute data includes the equipment serial number, category code, category name, equipment deployment location, fusion group, and connected equipment serial number.

3. The smart city sanitation drone real-time monitoring system based on multi-sensor fusion according to claim 1 is characterized in that: In the data fusion module: The data fusion module includes an interval boundary determination unit and a fusion group association unit, wherein: The interval boundary determination unit locates the interval where the sensor is located in three-dimensional space based on the sensor configuration information list and defines this interval or a created auxiliary interval as a boundary interval based on specific discrimination rules. The specific discrimination rules include signal sensitivity threshold discrimination, coverage threshold discrimination, and device distribution density discrimination. The auxiliary interval is created by expanding the edge of the original interval outward to form a ring interval with a preset width, with the expansion direction perpendicular to the spatial normal of the original interval. The fusion group association unit establishes a corresponding fusion group in three-dimensional space according to the fusion group information on the sensor configuration information list, and associates it with the boundary interval. A fusion group can contain multiple boundary intervals, and a boundary interval belongs to only one fusion group.

4. The smart city sanitation drone real-time monitoring system based on multi-sensor fusion according to claim 1 is characterized in that: In the trajectory planning module: The trajectory planning module includes an initial point automatic setting unit and a position data management unit, wherein: The automatic initial point setting unit supports automatically setting an initial track point at the center of the boundary interval of the fusion group. When the fusion group contains two boundary intervals, the initial track point is located on the boundary interval with the larger interval coverage. When the fusion group contains multiple boundary intervals, the initial track point is located on the boundary interval closest to the center. The initial track point refers to the centralized positioning point of all connected entities of the UAV track in the fusion group. The position data management unit adjusts the position of the set initial track point and edits the data, which includes the number and identification of the initial track point.

5. The smart city sanitation drone real-time monitoring system based on multi-sensor fusion according to claim 1 is characterized in that: In the state tracking module: The state tracking module includes a fusion group assembly unit and a state tracking execution unit, where: The fusion group assembly unit establishes classification groups in three-dimensional space based on the sensor configuration information list and adds the fusion groups to the corresponding classification groups. A classification group can contain multiple fusion groups, and a fusion group belongs to only one classification group. The UAV tracks of all fusion groups contained in a classification group are connected to the same connection entity. There is a one-to-one correspondence between a classification group and a connection entity. The connection entity model is loaded into the corresponding spatial position in the sensor collaborative infrastructure according to the actual position of the connection entity. The state tracking execution unit plans the flight path between the initial track points of the fusion group and the corresponding connection entities, marks the key nodes on the path as path control points in three-dimensional space, and then connects the initial track points, each path control point and the connection entity in sequence to generate a flight path model. It also supports referencing existing flight path models to complete all path planning.

6. The smart city sanitation drone real-time monitoring system based on multi-sensor fusion according to claim 1 is characterized in that: In the result output module: The result output module includes a path expansion diagram output unit and an equipment statistical report output unit, where: The path expansion diagram output unit expands the flight path model into a two-dimensional expansion diagram and outputs it, and marks the length data on each branch; The device statistical report output unit outputs a statistical report including device serial number, path length, location, and connected device serial number.

7. The smart city sanitation drone real-time monitoring system based on multi-sensor fusion according to claim 1 is characterized in that: In the calibration parameter calculation unit: when the fusion group contains three or more boundary intervals, the position of the initial track point is determined by calculating the average coordinates of the geometric centers of each boundary interval; the path length is measured as the shortest continuous path along the surface contour of the UAV track, with the starting point of the path being the center point of the UAV track deployment position and the end point being the center point of the initial track point.

8. The smart city sanitation drone real-time monitoring system based on multi-sensor fusion according to claim 5 is characterized in that: In the state tracking execution unit: the setting rule of the path control point is to select a feature point on the flight path at every preset distance, and the feature points include interval turning points, equipment avoidance points and space intersections; the reference condition of the existing flight path model is that the angle between the path direction and the current planned path does not exceed thirty degrees, and the path length difference is within a preset error range.

9. The smart city sanitation drone real-time monitoring system based on multi-sensor fusion according to claim 2 is characterized in that: In the device information collation unit: the basis for extracting the sensor configuration information list includes the drone size parameter table, sensor performance manual and deployment interface specification; the storage format of the attribute data is a structured table, including a serial number column, a category code column, a category name column, a deployment location column, a fusion group column and a connection device column.

Images