Road engineering inspection platform based on unmanned aerial vehicle

Abstract

The invention relates to the technical field of highway engineering, in particular to a highway engineering inspection platform based on an unmanned aerial vehicle, and the platform comprises a highway model module which is used for constructing a model before highway construction through the unmanned aerial vehicle and constructing a virtual model according to construction requirements; the work task management module is used for recording planned construction tasks and planned operation tasks; the inspection module is used for setting an inspection task and an inspection route so as to obtain a real-time inspection image through the unmanned aerial vehicle; the model updating module is used for updating the model before highway construction in the highway construction process so as to generate a highway model according to the construction progress; the management center comprises a construction management module which is used for matching and comparing the inspection image with a planned construction task to obtain the progress condition of highway construction; and the operation management module is used for identifying the hidden danger condition of the road according to the inspection image. The construction progress and safety of the road can be ensured, and safe and stable operation of the road is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of highway engineering technology, and in particular to a highway engineering inspection platform based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Highway engineering lifecycle inspection refers to the management process that ensures the safe and efficient operation of highways through systematic inspection activities at each stage of a highway project, from planning, design, construction, operation to maintenance.

[0003] Current highway engineering lifecycle inspection work mainly relies on manual inspections, traditional video surveillance, and single drone aerial photography, which suffers from core problems such as low efficiency, incomplete coverage, poor accuracy, and delayed response. Pre-construction site clearing and measurement are inefficient; during construction, earthwork measurement is difficult, and manual measurement is affected by terrain such as large elevation differences in mountainous areas, resulting in errors exceeding 15%; station number extraction relies on manual labor, which is time-consuming and prone to omissions. Safety inspections during construction are lagging, with manual inspections averaging less than 20 kilometers per day, long hazard discovery cycles, and emergency response times exceeding 30 minutes. Maintenance risks are high during the operation period; bridge inspections rely on equipment such as suspended platforms, road closures affect traffic, and coverage of high piers and mountainous areas is insufficient, making it easy to miss defects. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a highway engineering inspection platform based on unmanned aerial vehicles (UAVs), which can ensure the progress and safety of highway construction, as well as guarantee the safe and stable operation of highways.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A highway engineering inspection platform based on unmanned aerial vehicles (UAVs) includes a highway model module, a task management module, an inspection module, a model update module, and a management center.

[0007] The highway model module is used to construct a three-dimensional model of the highway. The highway model module acquires highway point cloud data through UAVs and processes the highway point cloud data to construct a pre-construction model of the highway before construction. The highway model module also constructs a virtual model according to construction requirements.

[0008] The task management module is used to store planned construction tasks and planned operation tasks.

[0009] The inspection module is used to set inspection tasks and inspection routes, and the inspection module controls the drone to take pictures of the highway according to the inspection tasks and inspection routes to obtain real-time inspection images.

[0010] The model update module is used for data acquisition by the inspection module to update the pre-construction model of the highway during the highway construction process, so as to generate a highway model according to the construction progress.

[0011] The management center is used for data acquisition of the highway model module, the work task management module, and the inspection module. The management center includes a construction management module and an operation management module. The construction management module is used to obtain the progress of highway construction by matching and comparing the inspection images with the planned construction tasks. The operation management module is used to identify potential hazards on the highway based on the inspection images.

[0012] Furthermore, the UAV surveyor measures the highway to obtain highway point cloud data, and the highway model module processes the point cloud data using Civil 3D to generate a pre-construction model of the highway with location coordinates. The model update module updates the pre-construction model of the highway to a highway model with location coordinates based on the data from the inspection module.

[0013] Furthermore, the inspection module includes an inspection task setting submodule and an inspection route setting submodule.

[0014] The inspection task setting submodule is used to acquire data for the planned construction task to obtain the required inspection location. The inspection task setting submodule marks the required inspection location in the highway model and obtains the inspection coordinates based on the markings in the highway model.

[0015] The inspection route setting submodule is used for data acquisition by the highway model module and the inspection task setting submodule, and the inspection route setting submodule determines the optimal cruise route based on the inspection coordinates and the highway model.

[0016] Furthermore, the inspection route setting submodule generates multiple patrol routes based on the inspection coordinates and terrain conditions, and obtains the optimal solution based on terrain threat cost calculation.

[0017] Formula (1)

[0018] in, To pass through the nodes from the starting point The total cost of reaching the destination. From the starting point to the node The actual path cost; From the starting point to the node The minimum possible cost; For nodes The cost of terrain threats; This refers to the terrain weighting coefficient, which is used to generate paths with different terrain adaptability by adjusting different terrain weighting coefficients.

[0019] Furthermore, the terrain threat cost includes slope threat cost and obstacle threat cost, and the slope threat cost is calculated as follows:

[0020] Formula (2)

[0021] in, Slope threatens costs; The current slope; The maximum permissible slope for drones;

[0022] Formula (3)

[0023] in, For the path to the The distance to each obstacle; The obstacle threat factor is determined based on the size and shape of the obstacle. This represents the total number of obstacles.

[0024] Furthermore, the inspection module also includes an aerial obstacle avoidance submodule, which is used to identify obstacles during aerial photography and adjust the drone's shooting angle and position. The obstacle avoidance steps of the aerial obstacle avoidance submodule include:

[0025] A1. Obstacles are captured in real-time by drones to dynamically update the obstacle data and to calibrate the coordinates and shape of the obstacles:

[0026] A2. Generate an anisotropic no-fly zone based on the coordinates and shape of the obstacle, and calculate the height of the obstacle;

[0027] A3. Based on the height of the heterogeneous no-fly zone and obstacles, voxel sampling is performed to generate an initial viewpoint. A safe viewpoint is generated based on the scene depth model, and the quality of the safe viewpoint is optimized and viewpoint compensation is performed to enable the UAV to fly and shoot from the safe viewpoint.

[0028] Furthermore, the construction management module includes a construction progress management submodule and a safety management submodule.

[0029] The construction progress management submodule analyzes the inspection images and compares them with corresponding parts of the virtual model to calculate the actual completion percentage of the task. It also performs semantic analysis on the planned construction tasks to determine the current completion percentage. Furthermore, it compares the actual completion percentage with the planned completion percentage to obtain a delay value. When the delay value exceeds a delay threshold, the submodule sends an anomaly alert and marks the error in both the virtual model and the highway model.

[0030] The safety management submodule is used to acquire the inspection images and mark construction workers who are not wearing safety helmets using a visual algorithm to obtain marked images. The safety management submodule matches the corresponding construction tasks and responsible personnel in the planned construction tasks based on the coordinate positions corresponding to the inspection images. The safety management submodule then sends the marked images to the terminals of the corresponding responsible personnel.

[0031] Furthermore, the construction progress management submodule is also used to adjust the construction progress, and the construction progress management submodule adjusts the construction progress based on the impact of the environment and the delay threshold:

[0032] Formula (4)

[0033] in, The adjusted construction schedule; The construction schedule before the adjustment; For the first The weights of each environmental factor; For the first The influence coefficient of each environmental factor; The percentage represents the delay threshold, which is adjusted according to different environments.

[0034] Furthermore, the operation management module includes a hazard identification submodule and a hazard monitoring submodule.

[0035] The hidden danger identification submodule is used to acquire the inspection image, and according to the location coordinates of the inspection image, it acquires key images in the inspection image. The regular inspection module processes the key images through a deep learning algorithm to obtain real-time suspected hidden danger images and their corresponding location data.

[0036] The hazard identification submodule collects suspected hazard data at the locations corresponding to the suspected hazard images using multiple sensors mounted on the drone. The hazard identification submodule then uses multi-sensor fusion to identify the suspected hazard data to obtain the final hazard image and marks the hazards in the final hazard image.

[0037] The operation management module also includes a hazard monitoring submodule. The hazard monitoring submodule is used to acquire data from the hazard identification submodule. The hazard monitoring submodule obtains the hazard location, hazard type, and hazard data through the final hazard image to calculate the hazard impact value. When the growth rate of the hazard impact value is greater than the growth threshold, the hazard monitoring submodule issues an alarm signal.

[0038] Furthermore, the operation management module also includes a hazard collaboration submodule, which is used to store historical handling schemes for highway hazards. The hazard collaboration submodule uses a semantic matching algorithm to match the hazard location, hazard type, and hazard data in the historical handling schemes to obtain similar hazard handling schemes, and the hazard collaboration submodule uses similar hazard handling schemes as collaborative schemes.

[0039] The beneficial effects of this invention are:

[0040] 1. Utilizing on-site image data from drones to construct a pre-construction model of a highway can assist in optimizing route selection and structural design. Simultaneously, during the planning phase, multiple alternative plans can be integrated into a 3D model for visualization, allowing for a direct comparison of the advantages and disadvantages of each plan. This addresses visual blind spots and spatial conflicts that are difficult to detect in traditional 2D design, reduces human error in traditional estimation methods, and provides accurate data for construction organization design and resource allocation. The work task management module can import planned construction and operation tasks, providing supporting data for subsequent construction and operation management. The model update module gradually completes and modifies the pre-construction model according to the construction progress during the highway construction process, forming a complete highway model. Under the management center's guidance and through drone-based inspection and monitoring, the progress and safety of highway construction are ensured, as well as the safe and stable operation of the highway.

[0041] 2. The patrol task setting submodule can determine the required highway nodes to be photographed according to the task requirements. Simultaneously, under the function of the patrol route setting submodule, different terrain-adaptive paths are generated based on the patrol coordinates and adjusted terrain weight coefficients. The optimal patrol route is selected as the path with the lowest total cost from the starting point through node n to the destination, thereby reducing the difficulty and improving the efficiency of patrols. The calculation of total cost incorporates slope threat cost and obstacle threat cost, improving the accuracy of the patrol route.

[0042] 3. Due to the complex environment of highway inspections, drones are easily obstructed by difficult-to-identify obstacles such as high-voltage lines and temporary facilities during filming. The aerial obstacle avoidance submodule utilizes a Scene Depth Model (SDSM) for obstacle avoidance and occlusion analysis. By generating and moving the filtered SDSM, it quickly determines whether the viewpoint is within an obstacle or in a poor signal area, thereby optimizing the safety and connectivity of the drone's path and avoiding collisions and line-of-sight obstruction. When encountering unmarked obstacles during drone patrol, the adaptive SDSM can also be updated in real time to adapt to environmental changes, addressing difficult-to-identify obstacles and enhancing obstacle avoidance capabilities and viewpoint connectivity. Attached Figure Description

[0043] Figure 1 This is a structural block diagram of a highway engineering inspection platform based on unmanned aerial vehicles (UAVs) according to a preferred embodiment of the present invention.

[0044] Figure 2 This is the login interface of a UAV-based highway engineering inspection platform according to a preferred embodiment of the present invention.

[0045] Figure 3 This is a data statistics interface of a UAV-based highway engineering inspection platform according to a preferred embodiment of the present invention.

[0046] Figure 4 This is the device control interface of a UAV-based highway engineering inspection platform according to a preferred embodiment of the present invention.

[0047] Figure 5 This is the load control interface of a UAV-based highway engineering inspection platform according to a preferred embodiment of the present invention.

[0048] Figure 6 This is a map annotation interface of a UAV-based highway engineering inspection platform according to a preferred embodiment of the present invention.

[0049] Figure 7 This is the device management interface of a UAV-based highway engineering inspection platform according to a preferred embodiment of the present invention.

[0050] Figure 8 This is the route creation interface of a UAV-based highway engineering inspection platform according to a preferred embodiment of the present invention.

[0051] Figure 9 This is the route setting interface of a highway engineering inspection platform based on unmanned aerial vehicles (UAVs) according to a preferred embodiment of the present invention.

[0052] In the diagram, 1-Highway Model Module, 2-Work Task Management Module, 3-Inspection Module, 31-Inspection Task Setting Sub-Module, 32-Inspection Route Setting Sub-Module, 33-Aerial Photography Obstacle Avoidance Sub-Module, 4-Model Update Module, 5-Management Center, 51-Construction Management Module, 511-Construction Progress Management Sub-Module, 512-Safety Management Sub-Module, 52-Operation Management Module, 521-Hazard Identification Sub-Module, 522-Hazard Monitoring Sub-Module, 523-Hazard Collaboration Sub-Module. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0055] Please see Figures 1 to 9 A preferred embodiment of the present invention provides a highway engineering inspection platform based on unmanned aerial vehicles (UAVs), comprising a highway model module 1, a work task management module 2, an inspection module 3, a model update module 4, and a management center 5.

[0056] Highway Model Module 1 is used to construct a 3D model of a highway. Highway Model Module 1 acquires highway point cloud data through a drone and processes the highway point cloud data to construct a pre-construction model of the highway. Highway Model Module 1 also constructs a virtual model according to construction requirements.

[0057] The task management module 2 is used to store planned construction tasks and planned operation tasks.

[0058] The inspection module 3 is used to set the inspection task and inspection route, and the inspection module 3 controls the drone to take pictures of the road according to the inspection task and inspection route to obtain real-time inspection images.

[0059] The model update module 4 is used to acquire data from the inspection module 3, so as to update the model before highway construction during the highway construction process and generate a highway model according to the construction schedule.

[0060] In this embodiment, the construction of a pre-construction model of the highway using on-site image data from UAVs can assist in optimizing route selection and structural design. Simultaneously, during the planning phase, multiple alternative schemes are integrated into the 3D model for display, allowing for a direct comparison of the advantages and disadvantages of each scheme. This solves visual blind spots and spatial conflicts that are difficult to detect in traditional 2D design, reduces human error in traditional estimation methods, and provides accurate data for construction organization design and resource allocation. The work task management module 2 can import planned construction tasks and planned operation tasks, thus providing supporting data for subsequent construction and operation management. The model update module 4 gradually completes and modifies the pre-construction model of the highway according to the construction progress during the construction process, forming the final highway model.

[0061] The drone surveyor measures the highway to obtain highway point cloud data, and the highway model module 1 processes the point cloud data through Civil3D to generate a pre-construction model of the highway with location coordinates. The model update module 4 updates the pre-construction model of the highway to a highway model with location coordinates based on the data from the inspection module 3.

[0062] In this embodiment, the inspection module 3 includes an inspection task setting submodule 31, an inspection route setting submodule 32, and an aerial photography obstacle avoidance submodule 33.

[0063] The inspection task setting submodule 31 is used to acquire data for planned construction tasks to obtain the required inspection locations. The inspection task setting submodule 31 marks the required inspection locations in the highway model and obtains the inspection coordinates based on the markings in the highway model.

[0064] The inspection route setting submodule 32 is used for data acquisition of the highway model module 1 and the inspection task setting submodule 31, and the inspection route setting submodule 32 determines the optimal cruise route based on the inspection coordinates and the highway model.

[0065] The inspection route setting submodule 32 generates multiple patrol routes based on inspection coordinates and terrain conditions, and obtains the optimal solution based on terrain threat cost calculation.

[0066] Formula (1)

[0067] in, To pass through the nodes from the starting point The total cost of reaching the destination. From the starting point to the node The actual path cost; From the starting point to the node The minimum possible cost, It is obtained through heuristic function calculation, which can be achieved by calculating the sum of vertical and horizontal distances on a grid map, making it suitable for structured environments; For nodes The cost of terrain threats; This is the terrain weight coefficient. By adjusting different terrain weight coefficients, paths with different terrain adaptability can be generated. The value can be selected according to the terrain conditions, and it is generally 0.1-0.3.

[0068] Terrain threat costs include slope threat costs and obstacle threat costs. The slope threat cost calculation method is as follows:

[0069] Formula 2

[0070] in, Slope threatens costs; The current slope; The maximum permissible slope for drones;

[0071] Formula 3

[0072] in, For the path to the The distance to each obstacle; The obstacle threat factor is determined based on the size and shape of the obstacle. This represents the total number of obstacles.

[0073] The patrol task setting submodule 31 can determine the highway nodes to be photographed according to the task requirements. Simultaneously, under the function of the patrol route setting submodule 32, different terrain-adaptive paths are generated based on the patrol coordinates and adjusted terrain weight coefficients. The optimal patrol route is selected as the path with the lowest total cost from the starting point through node n to the destination, thereby reducing the difficulty and improving the efficiency of the patrol. The calculation of the total cost incorporates slope threat cost and obstacle threat cost, improving the accuracy of the patrol route.

[0074] The aerial obstacle avoidance submodule 33 is used by the drone to identify obstacles during aerial photography and to adjust the drone's shooting angle and position. The obstacle avoidance steps of the aerial obstacle avoidance submodule 33 include:

[0075] A1. Obstacles are captured in real-time by drones to dynamically update the obstacle data and to calibrate the coordinates and shape of the obstacles:

[0076] A2. Generate anisotropic no-fly zones based on the coordinates and shapes of obstacles, and calculate the height of the obstacles;

[0077] A3. Voxel sampling is performed based on the height of the no-fly zone and obstacles to generate an initial viewpoint. A safe viewpoint is generated based on the scene depth model, and the quality of the safe viewpoint is optimized and the viewpoint is compensated so that the drone can fly and shoot from the safe viewpoint.

[0078] Due to the complex environment of highway inspections, drones are easily obstructed by difficult-to-identify obstacles such as high-voltage lines and temporary facilities during filming. The aerial obstacle avoidance submodule 33 utilizes a Scene Depth Model (SDSM) for obstacle avoidance and occlusion analysis. By generating and moving the filtered SDSM, it quickly determines whether the viewpoint is within an obstacle or in a poor signal area, thereby optimizing the safety and connectivity of the drone's path and avoiding collisions and line-of-sight obstruction. When encountering unmarked obstacles during drone patrol, the adaptive SDSM can also be updated in real time to adapt to environmental changes, addressing difficult-to-identify obstacles and enhancing obstacle avoidance capabilities and viewpoint connectivity.

[0079] The management center 5 is used for data acquisition from the highway model module 1, the work task management module 2, and the inspection module 3. The management center 5 includes a construction management module 51 and an operation management module 52. The construction management module 51 is used to obtain the progress of highway construction by matching and comparing inspection images with planned construction tasks. The operation management module 52 is used to identify potential hazards on the highway based on the inspection images.

[0080] Under the management center 5, and through drone inspection and monitoring, the progress and safety of highway construction are ensured, as well as the safe and stable operation of the highway.

[0081] In this embodiment, the construction management module 51 includes a construction progress management submodule 511 and a safety management submodule 512.

[0082] The construction progress management submodule 511 is used to analyze the inspection images and compare the inspection images with the corresponding parts of the virtual model to calculate the actual completion percentage of the task. The construction progress management submodule 511 performs semantic analysis on the planned construction tasks to determine the current completion percentage of the task plan. The construction progress management submodule 511 compares the actual completion percentage of the task with the planned completion percentage of the task to obtain the delay value. When the delay value is greater than the delay threshold, the construction progress management submodule 511 sends an abnormal reminder signal and marks it in the virtual model and the highway model.

[0083] The construction progress management submodule 511 is also used to adjust the construction progress. The construction progress management submodule 511 adjusts the construction progress based on environmental impacts and delay thresholds:

[0084]

[0085] in, The adjusted construction schedule; The construction schedule before the adjustment; For the first The weights of each environmental factor; For the first The influence coefficient of each environmental factor; The percentage represents the delay threshold, which is adjusted according to different environments.

[0086] In this embodiment, the relationship table between environmental factors and influence coefficients, weights, and delay thresholds is as follows:

[0087]

[0088] The construction progress management submodule 511 compares the actual captured construction images with the corresponding locations in the virtual model, and combines this with the planned construction tasks to accurately assess the completion status of the tasks, thereby facilitating adjustments to the construction plan by management personnel. Furthermore, this application can adjust the construction schedule based on environmental conditions, preventing safety accidents caused by construction workers rushing to meet deadlines, and minimizing the impact on the construction progress.

[0089] The safety management submodule 512 is used to obtain inspection images and mark construction workers who are not wearing safety helmets through visual algorithms to obtain marked images. Based on the coordinate position of the inspection image, the safety management submodule 512 matches the corresponding construction task and responsible personnel in the planned construction task and sends the marked image to the terminal of the corresponding responsible personnel.

[0090] The operation management module 52 includes a hazard identification submodule 521, a hazard monitoring submodule 522, and a hazard collaboration submodule 523.

[0091] The hazard identification submodule 521 is used to acquire inspection images and, based on the location coordinates of the inspection images, to acquire key images from the inspection images. The periodic inspection module 521 processes the key images using a deep learning algorithm to obtain real-time images of suspected hazards and their corresponding location data.

[0092] The hazard identification submodule 521 collects suspected hazard data at the locations corresponding to suspected hazard images using multiple sensors mounted on the drone. The hazard identification submodule 521 identifies the final hazard image by performing multi-sensor fusion on the suspected hazard data and marks the hazards in the final hazard image.

[0093] In this embodiment, the hazard identification submodule 521 first obtains suspected hazard images and their corresponding location data quickly and in real time through a deep learning algorithm, and then uses a multi-sensor fusion method to detect the complex environment at the location of the suspected hazard image, so as to accurately identify the hazard, thereby improving both identification efficiency and accuracy.

[0094] The hazard monitoring submodule 522 is used to acquire data from the hazard identification submodule 521. The hazard monitoring submodule 522 obtains the hazard location, hazard type, and hazard data through the final hazard image and calculates the hazard impact value. When the growth rate of the hazard impact value is greater than the growth threshold, the hazard monitoring submodule 522 issues an alarm signal.

[0095] Under the function of the hazard identification submodule 521, the location of the hazard can be continuously monitored. When the impact value of the hazard changes abruptly, it proves that there is an abnormality in the hazard, thereby reminding the staff to take action and prevent accidents from occurring.

[0096] The hazard collaboration submodule 523 is used for storing historical handling schemes for highway hazards. The hazard collaboration submodule 523 uses a semantic matching algorithm to match the location, type, and data of hazards in the historical handling schemes to obtain similar hazard handling schemes, and the hazard collaboration submodule 523 uses similar hazard handling schemes as collaborative schemes.

[0097] The hazard collaboration submodule 523 can provide collaborative solutions for hazard handling based on historical data, reducing the difficulty and efficiency of hazard handling.

[0098] This embodiment also includes a management terminal, which includes...

[0099] like Figure 2 and Figure 3 As shown, in the user login and statistics module, users enter a specified address through their browser, fill in their industry name, account, and password, and then click login to enter the system. After logging in, they enter the statistics page, which displays statistical data such as the total number of drones, the total number of pilots, the total number of airports, flight distance, and flight time, allowing users to have a comprehensive understanding of the platform's resource usage.

[0100] like Figure 4 , Figure 5 and Figure 6 As shown, the unmanned equipment management module allows users to view basic airport information and drone status; it supports remote airport debugging, which can only be performed when the aircraft is not in operation; and it provides a real-time airport live streaming function. Furthermore, there are no restrictions on the number of industry-specific drones and drone airports that can be connected, meeting the needs of large-scale inspection operations. Figure 4 For drone management: Real-time display of drone live video; dynamic display of drone data; support for one-click takeoff, flight control, and point-and-click flight. Figure 5For load control, it supports taking photos, recording videos, and zooming; controlling the live streaming direction of the load lens; providing wide-angle, zoom, and infrared mode switching; and supporting full-screen live streaming and resolution switching to meet the shooting and observation needs in different scenarios. The unmanned equipment management module also includes auxiliary functions, including a loudspeaker, gimbal settings, and video / photo recording operations. The nine-grid function displays online pilots and supports video playback, sharing, refreshing the organization, and switching between nine-grid / map displays. The map can display annotations, panoramas, and image layers, facilitating team collaboration and information sharing. Multi-channel live streaming supports playback of different numbers of live streaming videos of drone operations, airport monitoring videos, etc.; it supports generating push and corresponding pull stream addresses for live streaming on other devices, facilitating multi-party real-time monitoring during road and bridge inspections.

[0101] like Figure 6 As shown, in the map annotation module, after selecting a folder, you can add annotations and modify the annotation names; diamond-shaped annotations support sharing QR codes; you can modify the annotation names, colors, and delete annotation points, which facilitates marking and managing inspection areas. The map annotation module also allows for file import and export, supporting the import of files exported from the DJI platform; selecting a folder allows you to export files, facilitating data interaction with other systems.

[0102] like Figure 7 As shown, the device management module displays the device name, device serial number, firmware version, online status, and operation options for the remote controller, aircraft, airport, and corresponding drone.

[0103] like Figure 8 and Figure 9 As shown, the route library module includes importing routes, creating routes, and converting lines to routes. Importing routes involves clicking the import icon to select and upload a route file; once successful, it will be displayed in the route list. Creating routes involves entering the route name, selecting the corresponding airport, route type, aircraft, and model; configuring relevant parameters; and supporting the addition of waypoints and setting related actions. It also provides route management and planning capabilities, supporting various route planning and import functions. Converting lines to routes involves setting relevant parameters, clicking on map line markers to convert them into routes, and then configuring the parameters for each waypoint after generation.

[0104] AI training and recognition module:

[0105] AI Training: Upload images and labeled files for training; the server completes the recognition training. It provides an additional independent function module for training based on the YOLO model, enabling image data annotation and model training. It supports private model training and can customize AI model training for the specific needs of road and bridge inspection.

[0106] AI Recognition: Supports multiple target recognition and data transmission; configurable recognition types, confidence levels, alarm temperatures, and other parameters; the platform can control the AI ​​recognition function's on / off state. It provides various AI recognition and analysis capabilities. A single flight mission can simultaneously utilize two or more AI algorithms for recognition, improving the detection efficiency and accuracy of road and bridge inspections. Simultaneously, the AI ​​recognition image atlas module receives and organizes the AI ​​recognition results from each mission, uniformly storing and visually displaying the target object's recognition information, captured images, and related data, facilitating the organization and analysis of road and bridge inspection data. Furthermore, it supports precise setting of various parameters for different recognition tasks and allows users to independently upload pre-trained YOLO-based AI models.

[0107] Command Center Module:

[0108] The platform's homepage dashboard displays data statistics and early warning information; the left side shows statistics on equipment, pilots, and flight data, while the right side displays data related to clues.

[0109] The large screen in the clue center displays task data, a list of clues, and statistics on the number of suspected cases on the left, and patrol personnel, patrol numbers, and attendance on the right; it also supports viewing clue details and photos.

[0110] Dashboard and UI Customization: The dashboard includes daily updates, business statistics, inspection records, inspection plans, and statistics on data collection. UI customization is based on the system functions and is developed according to the user's statistical and display needs to meet the personalized needs of different users for data visualization and operation interface.

[0111] In this embodiment, the inspection steps during highway construction are as follows:

[0112] S1.1 Based on the planned construction tasks and through the inspection task setting submodule 31, the required inspection locations are marked in the highway model to obtain the coordinates of the required inspection locations;

[0113] S1.2 Generates multiple patrol routes based on inspection coordinates and terrain conditions, with routes starting from the origin and passing through nodes. The cruise route with the lowest total cost to reach the destination is the optimal cruise route. The drone is used to conduct inspections along the optimal cruise route and take the inspection images required for the mission.

[0114] S1.3 When obstacles exist in the captured image, the obstacles are dynamically updated, and their coordinates and shapes are calibrated; an anisotropic no-fly zone is generated based on the obstacle information, and the height of the obstacles is calculated; voxel sampling is performed based on the anisotropic no-fly zone and the height of the obstacles to generate an initial viewpoint; a safe viewpoint is generated based on the scene depth model, and the quality of the safe viewpoint is optimized and viewpoint compensation is performed so that the drone can fly and shoot from the safe viewpoint;

[0115] S1.4 Acquire inspection images, and update the model before highway construction to generate a highway model according to the construction progress.

[0116] S1.5 During construction progress management, the construction progress management submodule 511 analyzes the inspection images and compares them with the corresponding parts of the virtual model to calculate the actual completion percentage of the task. The construction progress management submodule 511 performs semantic analysis on the planned construction tasks to determine the current completion percentage of the task plan. The construction progress management submodule 511 compares the actual completion percentage of the task with the planned completion percentage to obtain a delay value. When the delay value exceeds the delay threshold, the construction progress management submodule 511 sends an abnormality alert signal and marks it in the virtual model and the highway model. When encountering abnormal environments, the construction progress is adjusted according to the environmental impact and the delay threshold.

[0117] S1.6 When performing safety management tasks, the inspection image is acquired, and the construction personnel who are not wearing safety helmets are marked by a visual algorithm to obtain the marked image. The safety management submodule 512 matches the corresponding construction task and the responsible personnel in the planned construction task according to the coordinate position of the inspection image. The safety management submodule 512 sends the marked image to the terminal of the corresponding responsible personnel.

[0118] This embodiment enables the following inspection steps during highway operation:

[0119] S2.1 Based on the planned construction tasks and through the inspection task setting submodule 31, the required inspection locations are marked in the highway model to obtain the coordinates of the required inspection locations;

[0120] S2.2 Generates multiple patrol routes based on inspection coordinates and terrain conditions, with routes starting from the origin and passing through nodes. The cruise route with the lowest total cost to reach the destination is the optimal cruise route. The drone is used to conduct inspections along the optimal cruise route and take the inspection images required for the mission.

[0121] S2.3 When obstacles exist in the captured image, the obstacles are dynamically updated, and their coordinates and shapes are calibrated; an anisotropic no-fly zone is generated based on the obstacle information, and the height of the obstacles is calculated; voxel sampling is performed based on the anisotropic no-fly zone and the height of the obstacles to generate an initial viewpoint; a safe viewpoint is generated based on the scene depth model, and the quality of the safe viewpoint is optimized and viewpoint compensation is performed so that the drone can fly and capture images from the safe viewpoint;

[0122] S2.4 During hazard identification, the hazard identification submodule 521 acquires key images from the inspection images based on their location coordinates. The periodic inspection module 521 processes these key images using a deep learning algorithm to obtain real-time suspected hazard images and their corresponding location data. The final hazard image is obtained by multi-sensor fusion of the suspected hazard data, and the hazards in the final hazard image are marked. The hazard monitoring submodule 522 obtains the hazard location, hazard type, and hazard data from the final hazard image and calculates the hazard impact value. When the growth rate of the hazard impact value exceeds a growth threshold, the hazard monitoring submodule 522 issues an alarm signal. The hazard collaboration submodule 523 uses a semantic matching algorithm to match the hazard location, hazard type, and hazard data in historical processing schemes to obtain similar hazard processing schemes. The hazard collaboration submodule 523 uses these similar hazard processing schemes as collaborative schemes.

Claims

1. A highway engineering inspection platform based on unmanned aerial vehicles (UAVs), characterized in that, It includes a highway model module (1), a work task management module (2), an inspection module (3), a model update module (4), and a management center (5). The highway model module (1) is used to construct a three-dimensional model of a highway. The highway model module (1) acquires highway point cloud data through a drone, and processes the highway point cloud data to construct a pre-construction model of the highway before construction. The highway model module (1) also constructs a virtual model according to construction requirements. The task management module (2) is used to store planned construction tasks and planned operation tasks; The inspection module (3) is used to set the inspection task and the inspection route, and the inspection module (3) controls the drone to take pictures of the highway according to the inspection task and the inspection route to obtain real-time inspection images. The model update module (4) is used for data acquisition by the inspection module (3) to update the pre-construction model of the highway during the highway construction process, so as to generate a highway model according to the construction progress. The management center (5) is used for data acquisition of the highway model module (1), the work task management module (2) and the inspection module (3), and the management center (5) includes a construction management module (51) and an operation management module (52). The construction management module (51) is used to obtain the progress of highway construction by matching and comparing the inspection images with the planned construction tasks; the operation management module (52) is used to identify the hidden dangers of the highway based on the inspection images.

2. The highway engineering inspection platform based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that: The UAV measures the highway to obtain highway point cloud data, and the highway model module (1) processes the point cloud data through Civil 3D to generate a pre-construction model of the highway with location coordinates. The model update module (4) updates the pre-construction model of the highway to a highway model with location coordinates based on the data of the inspection module (3).

3. The highway engineering inspection platform based on unmanned aerial vehicles (UAVs) according to claim 2, characterized in that: The inspection module (3) includes an inspection task setting submodule (31) and an inspection route setting submodule (32). The inspection task setting submodule (31) is used to acquire data for the planned construction task to obtain the required inspection location, and the inspection task setting submodule (31) marks the required inspection location in the highway model, and the inspection task setting submodule (31) obtains the inspection coordinates according to the markings in the highway model. The inspection route setting submodule (32) is used for data acquisition of the highway model module (1) and the inspection task setting submodule (31), and the inspection route setting submodule (32) determines the optimal cruise route based on the inspection coordinates and the highway model.

4. A highway engineering inspection platform based on unmanned aerial vehicles (UAVs) according to claim 3, characterized in that: The inspection route setting submodule (32) generates multiple patrol routes based on the inspection coordinates and terrain conditions, and the inspection route setting submodule (32) obtains the optimal solution based on terrain threat cost calculation: Official (1) in, To pass through the nodes from the starting point The total cost of reaching the destination. From the starting point to the node The actual path cost; From the starting point to the node The minimum possible cost; For nodes The cost of terrain threats; This refers to the terrain weighting coefficient, which is used to generate paths with different terrain adaptability by adjusting different terrain weighting coefficients.

5. A highway engineering inspection platform based on unmanned aerial vehicles (UAVs) according to claim 4, characterized in that: The terrain threat cost includes slope threat cost and obstacle threat cost, and the slope threat cost is calculated as follows: Official (2) in, Slope threatens costs; The current slope; The maximum permissible slope for drones; Official (3) in, For the path to the The distance to each obstacle; The obstacle threat factor is determined based on the size and shape of the obstacle. This represents the total number of obstacles.

6. A highway engineering inspection platform based on unmanned aerial vehicles (UAVs) according to claim 3, characterized in that: The inspection module (3) also includes an aerial obstacle avoidance submodule (33), which is used to identify obstacles during aerial photography and adjust the shooting angle and position of the drone. The obstacle avoidance steps of the aerial obstacle avoidance submodule (33) include: A1. Obstacles are captured in real-time by drones to dynamically update the obstacle data and to calibrate the coordinates and shape of the obstacles: A2. Generate an anisotropic no-fly zone based on the coordinates and shape of the obstacle, and calculate the height of the obstacle; A3. Based on the height of the heterogeneous no-fly zone and obstacles, voxel sampling is performed to generate an initial viewpoint. A safe viewpoint is generated based on the scene depth model, and the quality of the safe viewpoint is optimized and viewpoint compensation is performed to enable the UAV to fly and shoot from the safe viewpoint.

7. A highway engineering inspection platform based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that: The construction management module (51) includes a construction progress management submodule (511) and a safety management submodule (512). The construction progress management submodule (511) is used to analyze the inspection images, and the construction progress management submodule (511) matches and compares the inspection images with the corresponding parts of the virtual model to calculate the actual completion percentage of the task; the construction progress management submodule (511) performs semantic analysis on the planned construction task to determine the current time task completion percentage; the construction progress management submodule (511) compares the actual completion percentage of the task with the planned completion percentage of the task to obtain a delay value, and when the delay value is greater than the delay threshold, the construction progress management submodule (511) sends an abnormal reminder signal and marks it in the virtual model and the highway model; The safety management submodule (512) is used to obtain the inspection image and mark the construction workers who are not wearing safety helmets by using a visual algorithm to obtain the marked image. The safety management submodule (512) matches the corresponding construction task and the responsible person in the planned construction task according to the coordinate position of the inspection image. The safety management submodule (512) sends the marked image to the terminal of the corresponding responsible person.

8. A highway engineering inspection platform based on unmanned aerial vehicles (UAVs) according to claim 7, characterized in that: The construction progress management submodule (511) is also used to adjust the construction progress. The construction progress management submodule (511) adjusts the construction progress according to the impact of the environment and the delay threshold. Official (4) in, The adjusted construction schedule; The construction schedule before the adjustment; For the first The weights of each environmental factor; For the first The influence coefficient of each environmental factor; The percentage represents the delay threshold, which is adjusted according to different environments.

9. A highway engineering inspection platform based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that: The operation management module (52) includes a hazard identification submodule (521) and a hazard monitoring submodule (522). The hidden danger identification submodule (521) is used to acquire the inspection image, and according to the location coordinates of the inspection image, it acquires key images in the inspection image. The regular inspection module (521) processes the key images through a deep learning algorithm to obtain real-time suspected hidden danger images and their corresponding location data. The hazard identification submodule (521) collects suspected hazard data at the location corresponding to the suspected hazard image through multiple sensors mounted on the UAV. The hazard identification submodule (521) obtains the final hazard image by performing multi-sensor fusion on the suspected hazard data and marks the hazard in the final hazard image. The operation management module (52) also includes a hidden danger monitoring submodule (522). The hidden danger monitoring submodule (522) is used for data acquisition by the hidden danger identification submodule (521). The hidden danger monitoring submodule (522) obtains the hidden danger location, hidden danger type, and hidden danger data through the final hidden danger image to calculate the hidden danger impact value. When the growth rate of the hidden danger impact value is greater than the growth threshold, the hidden danger monitoring submodule (522) issues an alarm signal.

10. A highway engineering inspection platform based on unmanned aerial vehicles (UAVs) according to claim 9, characterized in that: The operation management module (52) also includes a hidden danger collaboration submodule (523), which is used to store the historical processing schemes of highway hidden dangers. The hidden danger collaboration submodule (523) uses a semantic matching algorithm to match the location of the hidden danger, the type of the hidden danger, and the data of the hidden danger in the historical processing schemes to obtain similar hidden danger processing schemes. The hidden danger collaboration submodule (523) uses similar hidden danger processing schemes as collaborative schemes.

Images