Highway cooperative monitoring method and system based on unmanned aerial vehicle and track robot

Abstract

The application relates to the technical field of data processing, and provides a highway cooperative monitoring method and system based on an unmanned aerial vehicle and a track robot, which solves the problems of weak dynamic deployment capability, lack of three-dimensional monitoring and data fusion defects of an existing system by means of a cooperation mechanism of timed inspection and fault type inspection, multifunctional signal point integration of a nest, function upgrading of a fixed camera signal transceiving function, combination of multi-source data fusion and network resource optimization, and improves real-time performance, accuracy and emergency response capability of highway monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a highway cooperative monitoring method and system based on unmanned aerial vehicles and rail robots. BACKGROUND

[0002] In the highway scene, the intelligent monitoring technology of unmanned aerial vehicles, rail robots and cameras has formed an efficient three-dimensional system. Unmanned aerial vehicles equipped with high-definition cameras, laser radars and other devices can automatically patrol complex areas such as bridges, tunnels and high slopes. The combination of unmanned aerial vehicles, nests and AI analysis can identify hidden dangers such as road debris and cracks through deep learning. Rail robots have the function of daily road inspection, and combined with unmanned aerial vehicles and camera intelligent devices for emergency disposal in tunnels, they cover the scenes of tunnel rescue and equipment maintenance, improve the efficiency of digital management, and build an intelligent operation and maintenance system for highways in the air, on the ground and at low altitude through the integration of 5G cloud platforms, edge computing and deep learning.

[0003] The current highway intelligent monitoring system usually uses unmanned aerial vehicles, rail robots and fixed cameras for cooperative operation, but still has the following shortcomings: lack of cooperation mechanism for regular inspection and fault inspection, unable to dynamically adjust monitoring resources according to preset periods or sudden accidents; fixed cameras only serve as image acquisition devices and do not integrate signal transceiver functions, resulting in the dependence of unmanned aerial vehicles and nests on a single communication link; nests are not used as signal points to cooperate with fixed cameras, making it difficult to realize AR navigation and multi-dimensional data interaction. SUMMARY

[0004] To overcome the shortcomings of the prior art, the present application provides a highway cooperative monitoring method and system based on unmanned aerial vehicles and rail robots, which solves the problems of weak dynamic resource adjustment capability, low communication link redundancy and difficult multi-dimensional data interaction caused by the lack of cooperation mechanism for regular and fault inspection, the lack of signal transceiver function in fixed cameras and the lack of cooperation between nests and cameras in the existing highway intelligent monitoring system.

[0005] To solve the above technical problems, the specific technical solutions of the present application are as follows:

[0006] In a first aspect, a highway cooperative monitoring method based on unmanned aerial vehicles and rail robots includes:

[0007] Step S1: deploying mobile unmanned aerial vehicle nests and rail robot systems to establish a cloud edge cooperative processing architecture;

[0008] Step S2: performing cooperative tasks of regular inspection and fault inspection, wherein regular inspection starts unmanned aerial vehicles and robots scanning based on a preset period, and fault inspection triggers the release of unmanned aerial vehicles by nests through environmental visibility thresholds or accident signals;

[0009] Step S3: Data calibration and interaction are implemented by a multi-source data fusion module, including real-time data exchange of the fixed camera as a communication node and the unmanned aerial vehicle and the nest;

[0010] Step S4: An automatic emergency response is started, including generation of an AR navigation instruction by the nest signal point and the fixed camera in cooperation;

[0011] Step S5: A road health assessment report is generated.

[0012] Further, the triggering condition of the fault type inspection in the step S2 of the expressway cooperative monitoring method based on the unmanned aerial vehicle and the track robot includes:

[0013] When the environmental visibility is lower than a preset threshold, the infrared induction charging platform of the nest is started to perform emergency charging on the unmanned aerial vehicle, and an enhanced data link is established through a dual-mode communication unit; if an accident signal is detected, the mechanical arm auxiliary device is controlled to perform precise release of the unmanned aerial vehicle.

[0014] Further, the data calibration and interaction in the step S3 of the expressway cooperative monitoring method based on the unmanned aerial vehicle and the track robot includes:

[0015] The millimeter wave radar data collected by the track robot and the three-dimensional laser scanning data of the unmanned aerial vehicle are spatiotemporally aligned, a continuous trajectory model of a moving target object is established through a cross-device tracking engine, and the exposure parameters of the thermal imaging camera are dynamically adjusted.

[0016] Further, the automatic emergency response of the step S4 of the expressway cooperative monitoring method based on the unmanned aerial vehicle and the track robot includes a hierarchical response mechanism:

[0017] A first-level response triggers adjacent unmanned aerial vehicle nests to form a monitoring array, a second-level response activates a self-cleaning bidirectional motion chassis to quickly reach an accident point, and a third-level response generates a three-dimensional diversion scheme and synchronously clears an emergency lane.

[0018] Further, the generation of the AR navigation instruction in the step S4 of the expressway cooperative monitoring method based on the unmanned aerial vehicle and the track robot includes:

[0019] Based on the cooperative positioning data of the nest signal point and the fixed camera, a dynamic traffic diversion path is generated, and an obstacle avoidance prompt is displayed through a variable pattern projection warning light group.

[0020] Further, the fixed camera as a communication node in the step S3 of the expressway cooperative monitoring method based on the unmanned aerial vehicle and the track robot includes:

[0021] The integrated microwave and cellular dual-mode communication module cooperates with the nest signal point to form a cellular communication network, and real-time transmission of unmanned aerial vehicle inspection data and robot scanning results.

[0022] Further, the highway cooperative monitoring method based on unmanned aerial vehicles and rail robots according to the present application, the road health assessment report generation of step S5 includes:

[0023] The monitoring efficiency analysis module fuses the pavement crack identification algorithm of the three-dimensional laser scanner and the visibility compensation algorithm, and outputs the road structure anomaly and maintenance suggestions.

[0024] The present application has the following advantages:

[0025] The present application significantly improves the dynamic deployment capability of highway monitoring resources through the cooperative mechanism of regular inspection and fault inspection. Regular inspection enables periodic investigation of daily hidden dangers, while fault inspection triggers an emergency response of the unmanned aerial vehicle through a visibility threshold or an accident signal, solving the coverage blind area problem caused by the single inspection mode of the traditional system. At the same time, the fixed camera is upgraded to a communication node and forms a cellular network with the nest, enhancing the communication link redundancy and avoiding data transmission interruption caused by single link failure.

[0026] Through the cooperative positioning of the nest signal point and the fixed camera, the system realizes the dynamic generation of AR navigation instructions and multi-dimensional data interaction. The three-dimensional diversion scheme reconstructs the roadblock space by combining laser radar point cloud and visible light image, accurately predicts the secondary accident risk area, and broadcasts the evacuation instructions through the sound field focusing array, significantly improving the emergency response efficiency. In addition, the road health assessment module fuses the crack identification and visibility compensation algorithm, providing a scientific basis for road maintenance and reducing the cost of manual inspection. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the drawings.

[0028] Fig. 1 The present application provides a highway cooperative monitoring method based on unmanned aerial vehicles and rail robots.

[0029] Fig. 2 The present application provides a highway cooperative monitoring method based on unmanned aerial vehicles and rail robots. DETAILED DESCRIPTION

[0030] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in connection with the embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application. The technical solutions provided by the embodiments of the present application are described in detail below in connection with the drawings.

[0031] In order to better understand the objects of the present application, the present application will be further described in detail below.

[0032] In the first aspect, please refer to Figs. 1-2 The expressway cooperative monitoring method based on the unmanned aerial vehicle and the track robot comprises the following steps:

[0033] Step S1: deploying a mobile unmanned aerial vehicle nest and a track robot system, and establishing a cloud edge cooperative processing architecture;

[0034] The mobile unmanned aerial vehicle nest is distributed along the expressway. An infrared induction charging platform and a dual-mode communication module are built in each nest, which supports emergency charging of the unmanned aerial vehicle and data transmission in bad weather. The track robot system is arranged on the central separation belt, and the track is integrated with a power supply line and an optical cable, so that real-time power supply and data transmission are realized. In the cloud edge cooperative processing architecture, the cloud is responsible for global data storage and complex algorithm operation, and the edge nodes (such as unmanned aerial vehicles and robots) perform real-time data processing and rapid response.

[0035] Step S2: performing cooperative tasks of timing inspection and fault type inspection;

[0036] The timing inspection starts the unmanned aerial vehicle and the robot to scan the whole road section based on a preset period (such as before the daily high-speed rail departure), covering key areas such as bridges and tunnels. The fault type inspection is triggered by an environmental visibility threshold (200 meters) or an accident sensor signal. After being triggered, the nest releases the unmanned aerial vehicle to perform an emergency task, and simultaneously starts a self-cleaning bidirectional motion chassis to quickly reach the accident point.

[0037] Step S3: multi-source data fusion and interaction;

[0038] The fixed camera is upgraded to a communication node, integrated with a microwave / cellular dual-mode communication module, and cooperates with the nest to form a cellular network. The millimeter wave radar data of the track robot and the three-dimensional laser scanning data of the unmanned aerial vehicle are spatio-temporally aligned through a cross-device tracking engine, a vehicle trajectory model is established, and the parameters of the thermal imaging camera are dynamically adjusted to adapt to the change of visibility.

[0039] Step S4: automatic emergency response and AR navigation;

[0040] After the accident triggers, the first level response releases the adjacent nest unmanned aircraft to form a monitoring array; the second level response activates the robot chassis to quickly reach the scene; the third level response generates a three-dimensional dredging scheme to synchronously empty the emergency lane. The AR navigation instruction is generated by the nest signal point and the camera cooperative positioning, and the dredging path is dynamically displayed by the variable pattern projection warning lamp group.

[0041] Step S5: Road health assessment report generation;

[0042] The pavement crack identification algorithm and the visibility compensation algorithm of the three-dimensional laser scanner are fused, the road structure abnormalities (such as cracks and settlement) are analyzed, a maintenance suggestion report is output, and highway maintenance decision is supported.

[0043] Embodiments of the present application;

[0044] Embodiment 1: System deployment and architecture establishment;

[0045] A mobile unmanned aircraft nest is deployed every 5 kilometers along the highway, and an infrared induction charging platform and a microwave / cellular dual-mode communication module are built-in each nest. The nest shell adopts lightning protection electromagnetic shielding design, and is equipped with a weather self-adaptive opening and closing mechanism to ensure safe operation in bad weather. The track robot system is arranged in the central separation belt, the track is integrated with a power supply line and an optical fiber data transmission channel, the robot is equipped with a liftable monitoring holder, and a thermal imaging camera and a laser radar are integrated. In the cloud edge cooperative processing architecture, the cloud server is responsible for storing global data and running a traffic flow prediction model, and the edge nodes (unmanned aircraft and robot) process local data in real time and are synchronized with the cloud through a 5G network.

[0046] Embodiment 2: Timely inspection and fault type inspection are cooperatively executed;

[0047] Timely inspection task: at 3 o'clock in the morning (preset period) every day, the unmanned aircraft automatically takes off, performs three-dimensional laser scanning on the bridge and tunnel at a cruising speed of 30km / h along the highway; at the same time, the track robot performs pavement crack scanning at a speed of 20km / h along the track, and the data is uploaded to the cloud in real time. Fault type inspection trigger:

[0048] Visibility trigger: when the visibility sensor detects that the visibility is less than 200 meters, the nest starts the infrared induction charging platform to urgently charge the unmanned aircraft (completed within 15 minutes), and establishes a data link with the monitoring center through the cellular network. After the unmanned aircraft takes off, it switches to the thermal imaging mode to scan the road surface.

[0049] Accident trigger: after the traffic accident sensor detects the collision signal, the nest control mechanical arm auxiliary device releases the unmanned aircraft, the unmanned aircraft locates the accident point through the AR label, synchronously activates the track robot self-cleaning bidirectional motion chassis, and reaches the scene within 5 minutes.

[0050] Example 3: Multi-source data fusion and communication coordination;

[0051] Fixed cameras are upgraded to communication nodes, integrated with microwave / cellular dual-mode modules, and form a cellular network with the nest. For example, when an accident occurs on a certain section of road:

[0052] Millimeter wave radar data (target speed, distance) collected by track robots and three-dimensional laser point cloud data from drones are aligned through cross-device tracking engines to generate continuous vehicle trajectory models.

[0053] Thermal imaging cameras automatically adjust exposure parameters based on visibility compensation algorithms to enhance image contrast in foggy conditions.

[0054] Cameras act as relay nodes, transmitting real-time high-definition video of the accident scene taken by drones to the emergency command center, while receiving traffic diversion instructions from the cloud.

[0055] Example 4: Automated emergency response and AR navigation;

[0056] Primary response: Drones are simultaneously released from drone nests within 3 kilometers of the accident point, forming an aerial monitoring array that captures panoramic videos of the accident site from multiple angles.

[0057] Secondary response: After the track robot arrives at the accident point, the laser radar mounted on the lifting gimbal scans the roadblock, generating a three-dimensional point cloud model, and the self-cleaning chassis clears the road debris.

[0058] Tertiary response:

[0059] The roadblock modeling component fuses laser radar data and camera visible light images to reconstruct the spatial position of the accident vehicle and generate a traffic diversion plan (e.g., close the left lane and guide vehicles to change lanes to the right).

[0060] AR navigation instructions are superimposed on emergency lanes through nest signal points and camera positioning data, and variable pattern projection warning light groups display red arrow pointers on the ground to guide detours.

[0061] Sound field focusing arrays broadcast multilingual voice instructions (such as "Accident ahead, please drive on the right") in a directional manner, avoiding interference from other areas.

[0062] Example 5: Road health assessment and maintenance decision;

[0063] Drones' three-dimensional laser scanners scan the road surface daily, and a crack length >5cm is automatically marked by a crack length identification algorithm, and a crack density index is calculated.

[0064] Visibility compensation algorithms analyze monitoring coverage under different weather conditions to generate monitoring effectiveness reports (e.g., effective monitoring distance reduced to 150 meters in foggy weather, requiring increased inspection frequency).

[0065] The system outputs a comprehensive evaluation report every month, prompting the maintenance department to repair the road section with crack density > 0.5 cracks per square meter in priority, and suggesting to add light supplementing equipment in low visibility areas.

[0066] The embodiment solves the problem that the traditional system cannot balance daily inspection and sudden accidents through the timing inspection and fault inspection cooperation mechanism. The fixed camera is used as a dual-mode communication node, and forms a cellular network with the nest, so that data transmission is not interrupted in an accident. The AR navigation instruction combines spatial positioning and dynamic projection to realize accurate traffic diversion. The road health evaluation integrates multi-source data to improve the scientific nature of maintenance.

[0067] Through the above embodiment, the present application systematically solves the problems of resource allocation, communication link and data interaction in highway monitoring, and significantly improves the safety and operation efficiency.

[0068] The present application sets a dual-mode cooperation mechanism of timing inspection (preset period) and fault inspection (visibility threshold / accident signal trigger) through step S2. The trigger logic of fault inspection is further refined, for example, when the visibility is lower than the threshold, the unmanned aerial vehicle emergency charging and data link enhancement are started, and the mechanical arm accurately releases the unmanned aerial vehicle. The three-level response mechanism (monitoring array, robot rapid arrival, three-dimensional diversion) further enhances the dynamic resource adjustment capability, so that the rapid response under sudden accidents is further strengthened.

[0069] Step S3 upgrades the fixed camera to a communication node, and the present application limits it to integrate microwave / cellular dual-mode communication module to form a cellular network with the nest. The real-time transmission of the inspection data of the unmanned aerial vehicle and the robot solves the problem of single communication link caused by the traditional camera only as an image acquisition device. The data calibration and cross-device tracking engine verify the cooperation ability of the camera as a communication node, so that the data transmission stability is improved.

[0070] Step S4 generates AR navigation instructions through the cooperation of nest signal points and cameras, and refines the instruction generation logic (dynamic path planning and warning light group prompt). The three-dimensional diversion scheme in the three-level response combines roadblock space reconstruction and vehicle trajectory anomaly analysis to realize multi-dimensional data interaction. The health evaluation module integrates crack identification and visibility compensation algorithm to further verify the accuracy of multi-dimensional data interaction, and provides reliable basis for road maintenance.

[0071] Through the inspection cooperation mechanism, camera signal function upgrade, nest cooperative AR navigation and other innovative designs, the present application systematically solves the three technical problems of weak dynamic resource adjustment capability, low communication link redundancy and difficult multi-dimensional data interaction, and significantly improves the real-time performance, accuracy and emergency response capability of highway monitoring.

Claims

1. A method for cooperative monitoring of a highway based on a UAV and a track robot, characterized in that, Comprise: Step S1: Deploy mobile unmanned aerial vehicle nest and track robot system, and establish cloud edge collaborative processing architecture; Step S2: Perform the cooperative task of timing inspection and fault type inspection, wherein the timing inspection starts the unmanned aerial vehicle and robot scanning based on the preset period, and the fault type inspection triggers the unmanned aerial vehicle to be released by the nest through the environmental visibility threshold or accident signal; Step S3: Implement data calibration and interaction through a multi-source data fusion module, including fixed cameras as communication nodes for real-time data exchange with unmanned aerial vehicles and nests; Step S4: Start automatic emergency response, including the cooperation of nest signal points and fixed cameras to generate AR navigation instructions; Step S5: Generate a road health assessment report; The triggering conditions of the fault type inspection in step S2 include: When the environmental visibility is lower than the preset threshold, the infrared induction charging platform of the nest is started to perform emergency charging on the unmanned aerial vehicle, and an enhanced data link is established through a dual-mode communication unit; if an accident signal is detected, the mechanical arm auxiliary device is controlled to perform precise release of the unmanned aerial vehicle; The data calibration and interaction in step S3 include: The millimeter wave radar data collected by the track robot and the three-dimensional laser scanning data of the unmanned aerial vehicle are spatiotemporally aligned, a continuous trajectory model of the moving target object is established through a cross-device tracking engine, and the exposure parameters of the thermal imaging camera are dynamically adjusted; The automatic emergency response of step S4 includes a hierarchical response mechanism: The first level response triggers adjacent unmanned aerial vehicle nests to form a monitoring array, the second level response activates a self-cleaning bidirectional motion chassis to quickly reach the accident point, and the third level response generates a three-dimensional diversion scheme and synchronously clears the emergency lane; The generation of the AR navigation instructions in step S4 includes: Based on the cooperative positioning data of the nest signal points and fixed cameras, a dynamic traffic diversion path is generated, and an obstacle avoidance prompt is displayed through a variable pattern projection warning light group; The fixed cameras as communication nodes in step S3 include: Integrate microwave and cellular dual-mode communication modules, cooperate with nest signal points to form a cellular communication network, and transmit unmanned aerial vehicle inspection data and robot scanning results in real time; The generation of the road health assessment report in step S5 includes: Fusion of the pavement crack recognition algorithm of the three-dimensional laser scanner and the visibility compensation algorithm of the monitoring efficiency analysis module, output of the road structure anomaly and maintenance suggestion.

Images