A method and system for ground-air coordinated patrol for urban environment patrol
By employing a ground-air collaborative inspection method, utilizing the coordinated operation of drones and unmanned vehicles, and combining it with an improved whale optimization algorithm, the problem of low efficiency and high safety risks in traditional urban environmental inspections has been solved, achieving efficient and real-time inspections in complex urban environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CIVIL AVIATION FLIGHT UNIV OF CHINA
- Filing Date
- 2026-05-13
- Publication Date
- 2026-06-09
Smart Images

Figure CN122170968A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of automatic inspection technology, and in particular to a ground-air collaborative inspection method and system for urban environmental inspection. Background Technology
[0002] Urban environmental patrols are a fundamental daily task in the refined management of cities. They refer to the regular or irregular inspections and supervision conducted by professionals on the city's appearance, environmental sanitation, public facilities, and green spaces. Their main purpose is to promptly identify and address issues such as street vending, exposed garbage, damaged facilities, illegal construction, and lack of greenery. Through methods such as foot patrols, vehicle patrols, or digital inspections, combined with grid management and smart platforms, problems can be quickly reported and addressed, thereby maintaining a clean, safe, and orderly urban environment.
[0003] Traditional urban environmental patrols mainly rely on manual inspections and fixed monitoring, which suffers from low efficiency, limited field of view, slow response, and high personnel safety risks. Existing technologies can employ drones or unmanned vehicles for single-method inspections; however, existing drone or unmanned vehicle inspection methods are limited by battery life, field of view, or accessibility, making it difficult to achieve comprehensive, real-time inspections in complex urban environments. Summary of the Invention
[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a ground-air collaborative inspection method and system for urban environmental inspection.
[0005] The objective of this invention is achieved through the following technical solution:
[0006] In a first aspect, this application discloses a ground-air coordinated patrol method for urban environmental inspection, comprising the following steps: A drone's sensors collect environmental data of the inspection area; the drone's onboard processor compares the collected environmental data with pre-stored baseline environmental data to identify environmental anomalies; the drone transmits the environmental anomaly information to a ground early warning station via a first communication module; the ground early warning station generates a dispatch instruction based on the received environmental anomaly information and transmits the dispatch instruction to an unmanned vehicle platform via a second communication module; the unmanned vehicle platform's onboard controller responds to the dispatch instruction, controlling the unmanned vehicle platform to move to the location corresponding to the environmental anomaly information to perform a ground verification task, and transmits the verification result back to the ground early warning station via a third communication module; finally, the ground early warning station transmits the received verification result to the drone.
[0007] Its beneficial effects are as follows: This application employs an improved whale optimization algorithm framework to iteratively search for a feasible path from the starting point to the endpoint with the optimal overall cost in a three-dimensional solution space containing threat areas, obstacles, and flight constraints. This leverages the flexibility and real-time capabilities of unmanned equipment to improve environmental inspection efficiency and reduce the risks of traditional manual inspections, while providing inspection information for unmanned vehicles and adapting to the complex inspection situations in cities.
[0008] Secondly, this application discloses a ground-air coordinated patrol system for urban environmental patrols, and a ground-air coordinated patrol method for performing the aforementioned urban environmental patrols, comprising a drone, an unmanned vehicle platform, and a ground early warning station; the drone includes sensors, an onboard processor, and a first communication module; the unmanned vehicle platform includes an onboard controller and a third communication module; the ground early warning station includes a control processor for generating scheduling instructions, and a second communication module that is communicatively connected to the first communication module and the third communication module. Attached Figure Description
[0009] Figure 1 This is a simplified schematic diagram illustrating the execution flow of a ground-air coordinated patrol method for urban environmental patrol according to some embodiments of this application;
[0010] Figure 2 This is a schematic diagram of the execution flow of an improved whale optimization algorithm according to some embodiments of this application;
[0011] Figure 3 This is a modular schematic diagram of a ground-air coordinated patrol system for urban environmental patrol according to some embodiments of this application;
[0012] Figure 4 This is a simplified schematic diagram of the drone operation process according to some embodiments of this application;
[0013] Figure 5 This is a simplified schematic diagram illustrating the operation process of an unmanned vehicle platform according to some embodiments of this application;
[0014] Figure 6 This is a simplified schematic diagram illustrating the workflow of a ground-based early warning station according to some embodiments of this application. Detailed Implementation
[0015] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0016] refer to Figure 1 - Figure 6 This application provides an embodiment of a ground-air collaborative patrol method and system for urban environmental patrol.
[0017] refer to Figure 1 and Figure 2 According to an embodiment of this application, a ground-air coordinated patrol method for urban environmental inspection includes the following steps: the sensors of a drone collect environmental data of the inspection area; the sensors include, but are not limited to, lidar and optical cameras, such as airborne high-definition cameras, for collecting environmental data.
[0018] The onboard processor of the drone compares the collected environmental data with pre-stored baseline environmental data to identify abnormal environmental information; the onboard processor is a computing device on the drone, such as an onboard computer, used to process data.
[0019] The drone transmits the environmental anomaly information to a ground early warning station via a first communication module, reporting the situation. The ground early warning station analyzes the anomaly and generates a dispatch command based on the received environmental anomaly information. This dispatch command is then transmitted to the unmanned vehicle platform via a second communication module. The onboard controller on the unmanned vehicle platform responds to the dispatch command, controlling the platform to move to the location corresponding to the environmental anomaly information to perform a ground verification task. The verification result is then transmitted back to the ground early warning station via a third communication module on the unmanned vehicle platform. Finally, the ground early warning station transmits the received verification result back to the drone. For example, the first, second, and third communication modules can be any type of communication device, such as a data transmission antenna. Furthermore, the unmanned vehicle platform can include a vehicle body, a controller, and communication modules, with the onboard controller controlling the unmanned vehicle's movement.
[0020] The following explanation will be based on a specific embodiment.
[0021] The UAV first performs problem initialization and coding, receiving the inspection task definition from the ground early warning station as input information, including the starting point. ,end Task point sequence Set of known threat areas (such as no-fly zones, high-rise buildings) And drone performance constraints, such as maximum range Maximum yaw Pitch angle Minimum turning radius Simultaneously, a continuous track is encoded into a high-dimensional parameter vector. For example, taking a typical encoding method as an example, a series of ordered three-dimensional coordinates of waypoints can be used:
[0022] ,in , represents the k-th waypoint. The paths between waypoints are smoothed by linear connection or spline curve, and n is the total number of waypoints.
[0023] Therefore, during drone inspections, environmental data can be collected using onboard high-definition cameras and LiDAR. The drone's onboard processor performs preliminary data processing, and any anomalies are sent back to the ground-based early warning station for further analysis. After analyzing the anomalies, the ground-based early warning station can autonomously generate solutions. The station then dispatches an unmanned vehicle platform to verify the anomalies and handle them urgently. The unmanned vehicle platform verifies the location of the anomaly and returns the results to the ground-based early warning station. If an anomaly is found, the unmanned vehicle handles it urgently; otherwise, the verification results are sent back to the ground-based early warning station. Upon completion of the overall mission, the ground-based early warning station collects and organizes the inspection information, checks the data, and re-activates the unmanned vehicle platform and drone.
[0024] In some embodiments, before the UAV's sensors collect environmental data of the inspection area, a path planning step performed by the ground early warning station or the UAV is included, as referred to... Figure 2 The path planning steps include:
[0025] S41. Initialize algorithm parameters and population, and set key algorithm parameters including population size. Maximum number of iterations Screw constant Initial value of convergence factor and final value Adaptive weight initial value and final value and the probability threshold for controlling the balance between exploration and development. It can be set to 0.5. The generation is performed randomly within the solution space or based on heuristic rules (such as considering the connection of task points). An initial flight path is established, generating the initial population. .
[0026] S42. For each individual in the population 'i' represents an intermediate variable, and the fitness evaluation function is called. Calculate its cost and initial fitness. Select the individual with the best fitness value (lowest cost) from the initial population and set it as the current global optimum. That is, the initial optimal solution.
[0027] S43. Perform the main iteration optimization loop, for the iteration algebra arrive Perform the following steps based on the current iteration number: Nonlinear update convergence factor:
[0028]
[0029] Update coefficient vector: , ,in , Given a random vector. Update the adaptive weights:
[0030]
[0031] in, It is a nonlinear decay adjustment index.
[0032] For each individual in the population Perform a position update and generate a random number. ,like ,and If the value is less than 1, then proceed to the development phase and execute a strategy close to the optimal solution: select an encirclement strategy with a probability of 0.5.
[0033]
[0034] Alternatively, a spiral update strategy can be chosen with a probability of 0.5:
[0035]
[0036] in, This is the logarithmic spiral distance scaling factor.
[0037] like Then, the exploration phase begins, and a random search is performed: a reference individual is randomly selected from the current population. Perform a random search:
[0038]
[0039] Then, regarding the updated position... Conduct constraint checks and corrections to ensure its physical feasibility, including:
[0040] Apply boundary constraints to ensure that the coordinates of all waypoints are within the mission space.
[0041] Specifically, in the boundary constraint processing stage, the system first determines whether a waypoint exceeds the hard spatial boundary frame jointly defined by the mission geographic boundary and airspace regulations. For boundary violations, the system implements hierarchical spatial correction: if a waypoint is within the hard boundary but outside its internal elastic buffer boundary, a penalty term proportional to the boundary violation distance is added to the fitness calculation as a soft constraint; if a waypoint exceeds the hard boundary frame, mandatory coordinate correction is performed—horizontal position violations are moved back into the boundary using a reflection method based on the boundary normal vector, and altitude violations are projected to the legal flight altitude limit. After completing the coordinate correction, the system further checks and repairs the flight segments between adjacent waypoints, ensuring that the corrected track remains continuous, smooth, and conforms to UAV kinematic constraints while strictly meeting all spatial control requirements through the insertion of new points and local B-spline smoothing.
[0042] Threat avoidance is implemented; if a flight segment enters a threat area, repulsive forces are applied to relevant waypoints for fine-tuning or high-penalty marking. Kinematic smoothing is performed to check if the segment curvature exceeds [a certain limit]. , To minimize the turning radius, smoothing is achieved by inserting or adjusting the position of points when the turning radius is insufficient or when there are abrupt changes in the heading / pitch angle. It is understandable that the currently generated candidate trajectory violates the kinematic feasibility constraints of the UAV and must be corrected to ensure its safe and stable execution. The core judgment criteria are based on a quantitative analysis of the trajectory geometry, mainly involving the following two calculable physical quantities: insufficient turning radius or abrupt changes in heading / pitch angle.
[0043] In the threat avoidance phase, the system first establishes a three-dimensional dynamic repulsion force field model for static threat sources within the inspection area, such as buildings, high-voltage power line towers, and temporary no-fly zones. The strength and effective range of this model are dynamically set according to the type of threat and a preset safety distance. When a flight segment is detected to intersect with the safety envelope of any threat source, the system calculates the penetration depth and the location of the penetration point. Based on the gradient direction of this point in the repulsion force field, it generates a repulsion correction vector with a direction away from the threat source and a magnitude proportional to the penetration depth. Subsequently, the system applies this vector to the penetration point and its adjacent flight path points to adjust their positions and immediately performs local B-spline smoothing on the affected flight path segment to ensure that the flight path after threat avoidance still meets the curvature constraints of the aircraft. At the same time, a high penalty term based on penetration depth and threat level is added to the fitness assessment, thereby systematically guiding flight path planning away from all threat areas at the population evolution level, achieving safe, smooth, and flyable global path generation.
[0044] Therefore, calculate the new position. fitness value .like Then use in the population replace Otherwise, retain the original individual. Traverse the current population and find the individual with the best fitness. If its fitness is better than the current global optimum... Then update .
[0045] S44. Finally, terminate and output the results when the maximum number of iterations is reached. Or, the improvement of the global optimal solution is less than a preset threshold over several consecutive generations. The iteration terminates when the global optimal parameter vector is obtained. Decode it and convert it into a series of waypoints.
[0046] In this embodiment, B-spline curves are used to interpolate and smooth waypoints to generate a final flight path that meets the requirements for continuous movement of the UAV. The optimized flight path is then output to the UAV flight control system as the execution path for this inspection mission.
[0047] It should be noted that the fitness evaluation function in this embodiment... It serves as a guide for algorithm optimization, and its calculation process is as follows:
[0048] Path length calculation:
[0049]
[0050] Threat cost calculation:
[0051]
[0052] in It is a threat penetration penalty function; the greater the penetration depth, the higher the penalty value. is the set of known threat areas, and j is an intermediate variable.
[0053] Smoothness cost calculation:
[0054]
[0055] in Penalize excessive changes in heading angle.
[0056] Altitude cost calculation:
[0057]
[0058] in, The desired flight altitude; to encourage flight at a safe altitude.
[0059] The overall fitness is calculated as follows:
[0060]
[0061] in , , , These are the weighting coefficients for each item. In different examples, the importance of the weights can be set according to task preferences; for example, increasing the weights if security is a greater priority. .
[0062] It is understood that the embodiments of this application improve the whale optimization algorithm by simulating the encirclement, spiral approximation and random search mechanisms in the intelligent predation behavior of whales, and efficiently explores and develops in the complex solution space, and finally outputs a UAV inspection track that achieves the best balance between path length, safety, smoothness and flight feasibility.
[0063] According to an embodiment of this application, a ground-air coordinated patrol system for urban environmental inspection is provided, with reference to... Figure 3 Understanding includes: drones, unmanned vehicle platforms, and ground-based early warning stations;
[0064] The unmanned aerial vehicle (UAV) includes an onboard processor, an onboard high-definition camera, a lidar, a GPS positioning device, and a first communication module. The unmanned vehicle platform includes an unmanned vehicle body, a power unit, an unmanned vehicle controller, an environmental detection device, and a third communication module. The ground early warning station interacts with the UAV and the unmanned vehicle platform via a second communication module. The onboard high-definition camera, lidar, and GPS positioning device are all connected to the onboard processor via output transmission lines, and all parameters and commands of the UAV are transmitted and received through the first communication module. The power unit, environmental detection device, and third communication module are all connected to the unmanned vehicle controller via data transmission lines, and all parameters and commands of the unmanned vehicle are transmitted and received through the third communication module.
[0065] The drone performs either initial inspection or non-initial inspection.
[0066] During the initial inspection, the airborne processor compares the collected environmental data with the pre-stored baseline environmental data. During subsequent inspections, the airborne processor updates the baseline environmental data to the environmental data collected in the previous inspection, and then compares the updated baseline environmental data with the collected environmental data.
[0067] Specifically, during the initial inspection, the airborne high-definition camera and the lidar transmit the sensed environmental data to the airborne processor via a data transmission line. The airborne processor processes and analyzes the transmitted environmental data using built-in algorithms, compares the analysis results with pre-imported baseline environmental data from the ground early warning station, and sends any abnormal environmental information back to the ground early warning station via the first communication module. Simultaneously, the inspection data is automatically saved. During subsequent inspections, the UAV continues to collect environmental data as in the initial inspection. The airborne processor compares the environmental information obtained in this inspection with the environmental information obtained in the previous inspection and the preset baseline environmental data. Data that differs from the preset information and data that differs from the previous inspection are sent back to the ground early warning station via the first communication module, and the information from this inspection is automatically saved.
[0068] After completing all inspection tasks, the drone will automatically return to its starting position and submit all environmental data and its own flight data during the inspection to the ground early warning station.
[0069] It is understood that the system in this application embodiment achieves a closed-loop task flow from aerial inspection to ground verification to central dispatch through the collaborative operation of UAVs and unmanned vehicle platforms. The algorithm is primarily used for global trajectory generation and dynamic replanning of a single UAV in complex urban environments, and is deeply integrated with the unmanned vehicle dispatch system to support ground verification and emergency response after anomaly identification. Furthermore, this system has a mechanism for comparing initial and non-initial inspection data, further improving inspection intelligence and response efficiency.
[0070] refer to Figure 4 and Figure 5 During operation, the ground-based early warning station first plans the overall inspection mission and assigns the task to drones for initial inspection. During the drone inspection, onboard high-definition cameras and lidar collect environmental data. The drone's onboard processor performs preliminary data processing, and environmental anomaly information is sent back to the ground-based early warning station for further anomaly analysis. After analyzing the environmental anomaly information, the ground-based early warning station can autonomously generate a solution or have a solution manually specified. The ground-based early warning station then dispatches an unmanned vehicle platform to verify the environmental anomaly information and handle any abnormal situations urgently. After verifying the location of the environmental anomaly, the unmanned vehicle platform returns the verification results to the ground-based early warning station. If an anomaly is found, the unmanned vehicle handles it urgently; otherwise, the verification results are sent back to the ground-based early warning station. After the overall mission is completed, the ground-based early warning station collects and organizes the inspection information and recalls the unmanned vehicle platform and drones.
[0071] Specifically, the UAV receives the inspection task issued by the ground early warning station and performs its own equipment status checks. During the inspection task, the UAV collects environmental data using its onboard high-definition camera and LiDAR, transmitting the collected data to the onboard processor via a data transmission line. The processor then performs data preprocessing. The preprocessed environmental anomaly information and the UAV's real-time status information are sent back to the ground early warning station via the first communication module. The ground early warning station returns the verification results of the anomaly data to the UAV and determines whether to continue the inspection task based on its completion status and the UAV's current status. When the UAV receives a return command from the ground early warning station, or when the current inspection task is completed or the equipment is unable to continue the inspection task, the UAV will automatically return to the designated location.
[0072] Figure 5 As shown, after receiving a verification command from the ground-based early warning station, the unmanned vehicle will automatically navigate to the abnormal location to verify the environmental anomaly information. Upon completion of the verification, the unmanned vehicle platform will transmit the verification results back to the ground-based early warning station via a third communication module, while simultaneously handling any abnormal situations. When the verification task is completed or a return command is received from the ground-based early warning station, the unmanned vehicle platform will automatically return to the designated location.
[0073] like Figure 6 As shown, in some embodiments, the ground-based early warning station can also generate inspection tasks through manual operation or autonomous decision-making, and send the generated inspection tasks to the drones via a second communication module, allowing the drones to execute the planned inspection tasks. If it receives abnormal environmental information from the drone, the ground-based early warning station will issue a verification task to the unmanned vehicle platform, which will then verify the abnormal data and send the verification results back to the drone. Simultaneously, the ground-based early warning station will make a comprehensive judgment based on the equipment's own condition and the current task completion progress, and recall the working equipment in a timely manner to ensure equipment safety while improving inspection efficiency.
[0074] Therefore, during the initial inspection, the UAV processes environmental data collected by the lidar and high-definition camera in real time using its onboard processor, compares it with preset benchmark environmental data, and identifies abnormal areas. During subsequent inspections, the onboard processor compares the current data with historical inspection data and preset data to identify environmental changes and new anomalies. It then transmits the environmental anomaly information back to the ground early warning station via the first communication module, achieving a closed-loop response for the mission. Furthermore, upon receiving dispatch instructions from the ground early warning station, the unmanned vehicle platform automatically navigates to the anomaly point for on-site verification. It collects on-site data using environmental detection devices, processes it through the unmanned vehicle controller, and transmits it back to the ground early warning station, supporting anomaly verification and emergency response.
[0075] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.
Claims
1. A ground-air coordinated patrol method for urban environmental inspection, characterized in that, Includes the following steps: The drone's sensors collect environmental data from the inspection area; The onboard processor of the drone compares the environmental data collected this time with the pre-stored baseline environmental data to identify abnormal environmental information. The drone transmits the environmental anomaly information to the ground early warning station via the first communication module; The ground early warning station generates a dispatch instruction based on the received environmental anomaly information, and sends the dispatch instruction to the unmanned vehicle platform through the second communication module of the ground early warning station; The vehicle controller on the unmanned vehicle platform responds to the scheduling command, controls the unmanned vehicle platform to move to the location corresponding to the environmental anomaly information to perform the ground verification task, and transmits the verification result back to the ground early warning station through the third communication module on the unmanned vehicle platform; Then, the received verification results are sent to the UAV via the ground early warning station.
2. The ground-air coordinated patrol method for urban environmental patrol according to claim 1, characterized in that, The drone performs either initial inspection or non-initial inspection. During the initial inspection, the airborne processor compares the collected environmental data with the pre-stored baseline environmental data. During the non-initial inspection, the airborne processor updates the baseline environmental data to the environmental data collected in the previous inspection, and compares the updated baseline environmental data with the environmental data collected this time.
3. The ground-air coordinated patrol method for urban environmental patrol according to claim 1 or 2, characterized in that, Before the UAV's sensors collect environmental data of the inspection area, a path planning step is also included, performed by the ground early warning station or the UAV. The path planning step includes the following sub-steps: S41. Initialize the track population, where each track is represented by a series of waypoint parameters; S42. Calculate the fitness value of each track in the population; S43. Perform iterative optimization. In each iteration, based on the updated convergence factor and the generated random number, adaptively select the prey encirclement strategy, spiral update strategy, or random search strategy to update the individual positions in the track population, and recalculate the fitness value. S44. After the iteration is completed, the track with the best fitness is output as the inspection track.
4. The ground-air coordinated patrol method for urban environmental patrol according to claim 3, characterized in that, Calculating the fitness value for each track in the population includes: Path length L calculation: in, , Here, k represents three-dimensional coordinates, k is an intermediate variable, and n is the total number of waypoints. and Indicates the k-th and (k-1)-th waypoints; Threat cost T calculation: in It is a threat penetration penalty function. It is the set of known threat areas, and j is an intermediate variable; Smoothness cost S calculation: in, For the (k-1)th waypoint, The smoothness cost calculation function penalizes excessive changes in heading angle; Altitude cost A calculation: in, To achieve the desired flight altitude, flight at a safe altitude is encouraged; The overall fitness is calculated as follows: in, It is a high-dimensional parameter vector. ; , , , These are the weighting coefficients for each item.
5. The ground-air coordinated patrol method for urban environmental patrol according to claim 3, characterized in that, After each recalculation of fitness values and update of individual positions in the track population, constraints are applied to the updated individual positions in the track population, including: Apply boundary constraints; Threat avoidance measures are implemented. If a flight segment enters a threat area, a repulsive force is applied to the relevant waypoints for fine-tuning or marking them with high penalties. Perform kinematic smoothing and check if the segment curvature exceeds [the specified limit]. , To achieve the minimum turning radius, smoothing is achieved by inserting or adjusting the position of the turning point when the turning radius is insufficient or when there are sudden changes in the heading / pitch angle.
6. The ground-air coordinated patrol method for urban environmental patrol according to claim 1 or 2, characterized in that: After the UAV completes its inspection mission or receives a return command from the ground early warning station, the system controls the UAV to automatically return to its starting position and uploads mission data to the ground early warning station via the first communication module.
7. A ground-air coordinated patrol system for urban environmental inspection, characterized in that, The ground-air coordinated patrol method for performing urban environmental patrols as described in any one of claims 1-6 includes unmanned aerial vehicles, unmanned vehicle platforms, and ground early warning stations; The drone includes sensors, an onboard processor, and a first communication module; The unmanned vehicle platform's onboard controller and third communication module; The ground early warning station includes a control processor for generating dispatch instructions, and a second communication module that is communicatively connected to the first communication module and the third communication module.
8. The ground-air coordinated patrol system for urban environmental patrol according to claim 7, characterized in that, The airborne processor includes a path planning module, which is configured to execute a path planning step, the path planning step including the following sub-steps: S41. Initialize the track population, where each track is represented by a series of waypoint parameters; S42. Calculate the fitness value of each track in the population; S43. Perform iterative optimization. In each iteration, based on the updated convergence factor and the generated random number, adaptively select the prey encirclement strategy, spiral update strategy, or random search strategy to update the individual positions in the track population, and recalculate the fitness value. S44. After the iteration is completed, the track with the best fitness is output as the inspection track.
9. The ground-air coordinated patrol system for urban environmental patrol according to claim 7, characterized in that, The sensors include lidar and optical cameras.