Method and system for automatically splitting and recombining inspection route of unmanned aerial vehicle

Abstract

The invention discloses an unmanned aerial vehicle inspection route automatic splitting and recombining method and system. The method is based on cable-stayed bridge structural members. An unmanned aerial vehicle carrying a multi-sensor system is utilized to complete image and multi-dimensional data acquisition of a cable-stayed bridge main tower, a stay cable and a bridge deck beam according to a preset route, and the images and the multi-dimensional data are transmitted to a cloud end in real time; the cloud analyzes the image and the multi-dimensional data in real time, identifies an abnormal area of the cable-stayed bridge structural member and marks the abnormal area; based on the marking result of the abnormal area, combining a three-dimensional model of the cable-stayed bridge and structural part segmentation logic, dynamically splitting the preset route into independent sub routes, and generating a review instruction; generating a local optimal path by using a dynamic path planning algorithm, and controlling the unmanned aerial vehicle to execute a recheck task of the sub-route to generate a recheck result; and generating a cable-stayed bridge structural member inspection report in combination with the reinspection result, and updating an abnormal region database. The abnormal area review efficiency is improved, and accurate and efficient inspection is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle (UAV) bridge inspection, and in particular to a method and system for automatic splitting and recombining of UAV inspection routes. BACKGROUND

[0002] With the wide application of UAV technology in bridge inspection, the traditional inspection method covers the entire area of the bridge by presetting a fixed route. However, there are significant defects in actual operation. When detecting abnormalities in cable-stayed bridge components (such as component damage, main tower cracks, and cable corrosion), the traditional method needs to repeatedly execute the complete route for review, resulting in a large amount of invalid flight time, reducing the inspection efficiency, and wasting the UAV's endurance resources. In the prior art, bridge inspection systems focus on image recognition and crack detection algorithms, and lack effective solutions for dynamic adjustment of inspection routes. They cannot intelligently optimize the inspection path based on real-time detection results, and are difficult to meet the efficient inspection needs of complex structure bridges such as cable-stayed bridges. Therefore, there is an urgent need for a method and system that can dynamically split and recombine routes based on cable-stayed bridge component anomaly detection results to solve the problem of low efficiency of traditional inspection.

[0003] It should be noted that the information disclosed in the above background section is only for understanding the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0004] The present application provides a method and system for automatic splitting and recombining of UAV inspection routes, which can solve at least one problem in the background art.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solutions: A method for automatic splitting and recombining of UAV inspection routes based on cable-stayed bridge components; a UAV equipped with a multi-sensor system is used to complete image and multi-dimensional data collection of the main tower, cable and bridge deck of the cable-stayed bridge according to a preset route, and the image and multi-dimensional data are transmitted to the cloud in real time; the cloud analyzes the image and multi-dimensional data in real time, identifies and marks the abnormal areas of the cable-stayed bridge components, and the marking results include the abnormal position, type and severity; based on the marking results of the abnormal areas, combined with the three-dimensional model of the cable-stayed bridge and the component segmentation logic, the preset route is dynamically split into independent sub-routes, and a review instruction containing the spatial range, execution order and priority is generated; using a dynamic path planning algorithm, a local optimal path is generated according to the priority of the sub-route, the real-time state of the UAV and the environmental data, the UAV is controlled to perform the sub-route review task to generate a review result; a cable-stayed bridge component inspection report is generated combined with the review result, and the abnormal area database is updated.

[0006] Preferably, the generation of the preset flight path comprises the following steps: based on the three-dimensional model of the cable-stayed bridge, the main tower, the cable-stayed cable and the bridge deck are divided into a plurality of key detection areas; according to the geometric characteristics and detection requirements of each key detection area, the preset flight path covering the whole bridge is generated, and the detection parameters of each key detection area are predefined, including flight height, shooting angle and sensor configuration.

[0007] Preferably, the cloud AI deep recognition model is used for real-time analysis of the image and the multi-dimensional data, specifically including: using a multi-branch first CNN model adopted by the cloud AI deep recognition model, judging whether there is a potential crack in the current view according to the collected image, and if so, marking the location of the potential crack in the view and the preset color and format; each branch of the first CNN model trains and judges whether there is a potential crack in the cable-stayed bridge; using a multi-branch second CNN model adopted by the cloud AI deep recognition model, judging whether there is a crack in the marked position according to the color and format in the view and the branch adaptively matched; each branch of the second CNN model trains and judges whether there is a crack in the marked position and the crack position and the crack severity in the image; the input of the cloud AI deep recognition model fuses the image and the laser radar point cloud data obtained by the multi-sensor; the output includes abnormal position coordinates, abnormal type label and abnormal severity score.

[0008] Preferably, the abnormal severity score is a dynamic threshold: if the abnormal area is located at a key connection part of the main tower or the cable-stayed cable, the preset severity threshold is automatically reduced; the key connection part includes the main tower, the main beam and the cable-stayed cable of the cable-stayed bridge; when the abnormal area is located at the key connection part, the preset severity threshold is automatically reduced by 30%-50%.

[0009] Preferably, the preset flight path is dynamically split into independent sub-flight paths, which comprises: according to the distribution of the abnormal area, the preset flight path is split into the sub-flight path covering the abnormal area and the reserved baseline flight path; the sub-flight path is assigned a priority, the priority is calculated based on the abnormal severity score and the structure importance weight, the priority calculation model: priority P = α × abnormal severity score S + β × structure importance weight I; wherein, the abnormal severity score S: the value range [0, 10] is output by the cloud AI deep recognition model, including crack width and rust area proportion quantitative indicators; the structure importance weight I: determined according to structural mechanics analysis, the value range [0, 1]; α and β are weight coefficients.

[0010] Preferably, the dynamic path planning algorithm determines the problem waypoints, the distances between the problem waypoints, and the priorities of the sub-routes by the abnormal area, generates a sub-route for re-inspection of the key points; searches for a minimum connected subgraph containing the target area with the abnormal point as the core, ensures that the sub-route covers the abnormal area and its adjacent key sections, and eliminates irrelevant points; each point contains three-dimensional coordinates (x, y, z), attitude angles (yaw angle ψ, pitch angle θ), and residence time t; according to the structure segmentation logic, automatically adds the upper and lower adjacent points as the protection area; checks whether the sub-route contains a complete detection angle of view, and if there is a missing angle of view, supplements the necessary points from the preset route; optimizes the order of the points in the sub-route, clusters all the points, and generates the locally optimal path.

[0011] Preferably, the unmanned aerial vehicle is equipped with the multi-sensor system, which collects environmental lighting data in real time and transmits it to the cloud; the cloud generates wavelength adjustment instructions according to the environmental lighting data to control the light source of the unmanned aerial vehicle to emit light of a specific wave band matching the material of the cable-stayed bridge.

[0012] Preferably, the frequency of the same position abnormality in the inspection report is counted, and if the frequency exceeds a preset threshold, the position is marked as a key monitoring point, and the priority of the sub-route of the key monitoring point is automatically increased in subsequent inspections.

[0013] Preferably, it further comprises: constructing a digital twin model of the structure of the cable-stayed bridge in the cloud, synchronizing the re-inspection results of the unmanned aerial vehicle with the three-dimensional model data in real time, and dynamically displaying the abnormal area and its development trend through picture comparison.

[0014] The application further provides an unmanned aerial vehicle inspection route automatic splitting and recombination system, which realizes the method described in any of the above, and comprises: a multi-sensor data acquisition module, which is used for completing image and multi-dimensional data acquisition of a main tower, a stay cable and a bridge deck beam of a cable-stayed bridge according to a preset route, and transmitting the image and the multi-dimensional data to the cloud in real time; an anomaly detection module, which is used for performing real-time analysis on the image and the multi-dimensional data, identifying and marking an abnormal area of the cable-stayed bridge structure, and marking results including marking an abnormal position, type and severity; a route dynamic management module, which is used for dynamically splitting the preset route into independent sub-routes based on the marking results of the abnormal area, combining a three-dimensional model of the cable-stayed bridge and a structure segmentation logic, and generating a review instruction including a spatial range, an execution order and a priority; a path planning engine module, which is used for generating a locally optimal path according to the priority of the sub-route, real-time state of an unmanned aerial vehicle and environmental data by using a dynamic path planning algorithm, controlling the unmanned aerial vehicle to perform a review task of the sub-route and generating a review result; and a report generation module, which is used for generating a cable-stayed bridge structure inspection report in combination with the review result and updating an abnormal area database.

[0015] The application has the following beneficial effects: The application obtains data by the unmanned aerial vehicle loaded with the multi-sensor system, analyzes the data by the cloud to obtain and mark an abnormal area, further splits an original route, generates a sub-route task queue and a priority instruction, generates a locally optimal review path based on a route planning algorithm, integrates review data and outputs a structured inspection report, realizes accurate positioning and efficient review of the abnormal area, significantly improves detection efficiency and reliability of a bridge key structure, reduces invalid flight time, improves review efficiency of the abnormal area, and realizes accurate and efficient inspection of the cable-stayed bridge key structure. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 FIG. 1 is a schematic diagram of an unmanned aerial vehicle inspection route automatic splitting and recombination method according to an embodiment of the application.

[0017] Figure 2 FIG. 2 is a schematic diagram of an unmanned aerial vehicle inspection route automatic splitting and recombination system according to an embodiment of the application. DETAILED DESCRIPTION

[0018] The embodiments of the application are described in detail below. It should be emphasized that the following description is only exemplary and is not intended to limit the scope of the application and its applications, and the embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0019] It is to be noted that when an element is referred to as being "fixed" or "set" on another element, it can be directly on the other element or indirectly on the other element, with one or more intervening elements. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or indirectly connected to the other element by way of one or more intervening elements. Further, the connection can be for fixed or coupling or communicational purposes.

[0020] It is to be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like, specify relative positions or orientations based on the orientations or positions shown in the drawings, and are used for convenience of description and simplicity of description, and thus cannot be construed as indicating or implying that a device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be construed as limiting the present application.

[0021] In addition, the terms "first", "second", "third", etc. are used only for descriptive purposes and should not be construed as indicating or implying relative importance or a required number thereof. Thus, features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0022] Referring to Figure 1 One embodiment of the present application provides a method for automatically splitting and recombining inspection routes of unmanned aerial vehicles, based on structural components of a cable-stayed bridge; An unmanned aerial vehicle equipped with a multi-sensor system is used to collect images and multi-dimensional data of the main tower, stay cables and bridge deck of the cable-stayed bridge according to a preset route, and the images and multi-dimensional data are transmitted to the cloud in real time; The cloud analyzes the images and multi-dimensional data in real time, identifies and marks abnormal areas of the structural components of the cable-stayed bridge, and the marking results include the abnormal positions, types and severity; Based on the marking results of the abnormal areas, combined with a three-dimensional model of the cable-stayed bridge and a segmentation logic of the structural components, the preset route is dynamically split into independent sub-routes, and review instructions including spatial range, execution order and priority are generated; Using a dynamic path planning algorithm, a locally optimal path is generated according to the priority of the sub-routes, the real-time state of the unmanned aerial vehicle and environmental data, and the unmanned aerial vehicle is controlled to perform the review task of the sub-routes to generate review results; Combined with the review results, a patrol report of the structural components of the cable-stayed bridge is generated, and an abnormal area database is updated.

[0023] The present application realizes accurate positioning and efficient re-inspection of abnormal areas by means of multi-sensor data fusion, cloud-based deep analysis, dynamic route optimization and other core technologies, significantly improves the detection efficiency and reliability of key structural components of the bridge, reduces the invalid flight time, improves the re-inspection efficiency of the abnormal area, and realizes accurate and efficient inspection of the key structural components of the cable-stayed bridge.

[0024] In one specific embodiment of the present application, the multi-sensor system includes a visible light camera, a laser radar, and an infrared thermal imager.

[0025] In one embodiment of the present application, a preset route is first set, and a flight path of the preset route is set, covering all designated bridge areas. The unmanned aerial vehicle carrying a composite sensor system (visible light camera, laser radar, infrared thermal imager) takes off from the ground control station, and inspects the bridge according to the preset route.

[0026] In one embodiment of the present application, the unmanned aerial vehicle carrying a camera completes real-time image acquisition of the bridge to be detected according to the flight route of the preset route, and transmits the acquired images to the cloud in real time; activates the multi-sensor system carried on the unmanned aerial vehicle, including a high-definition camera, a laser radar, an infrared camera and a hyperspectral sensor; uses the unmanned aerial vehicle carrying a high-resolution visible light camera, a high-precision laser radar, an infrared thermal imager and a hyperspectral sensor to form a multi-dimensional detection system. According to the structural characteristics of the cable-stayed bridge, fine interval acquisition strategy is adopted for key areas such as main tower, stay cable and bridge deck beam to ensure coverage of most key structural components.

[0027] Based on large-scale labeled sample training, the collected data are analyzed to support deep analysis and accurate classification of abnormal areas; intelligent judgment of key parts in combination with preset key parts in the bridge model database classifies the marked positions according to importance, and automatically adjusts the detection sensitivity of the key area.

[0028] In one embodiment of the present application, the generation of the preset route includes the following steps: Based on the three-dimensional model of the cable-stayed bridge, the main tower of the cable-stayed bridge, the stay cable and the bridge deck are divided into a plurality of key detection areas; According to the geometric characteristics and detection requirements of each key detection area, the preset route covering the whole bridge is generated, and the detection parameters of each key detection area are predefined, including flight height, shooting angle and sensor configuration.

[0029] In another embodiment of the present application, the images and the multi-dimensional data are analyzed in real time by a cloud AI deep recognition model, specifically including: The multi-branch first CNN model adopted by the cloud AI deep recognition model adaptively selects a matched branch according to the collected image to determine whether the view at the current time exists a potential crack, and if so, marks the location of the potential crack in the view and a preset color and format mark; each branch in the first CNN model trains and judges whether the cable-stayed bridge has a potential crack; The multi-branch second CNN model adopted by the cloud AI deep recognition model adaptively matches a branch according to the color and format in the view to determine whether the marked position exists a crack and the crack position and crack severity; each branch of the second CNN model trains and judges whether the marked position in the image exists a crack and the crack position and crack severity; The input of the cloud AI deep recognition model fuses the image and laser radar point cloud data obtained by the multi-sensor; and the output includes an abnormal position coordinate, an abnormal type label and an abnormal severity score.

[0030] In a specific embodiment, the unmanned aerial vehicle is connected to the ground airport and the cloud service platform for communication, and completes real-time image collection of the bridge to be detected according to a preset flight route, and transmits the collected image to the cloud service platform in real time. The visible light camera collects the bridge surface texture at multiple angles in high definition through automatic zoom technology; the data alignment of each sensor is realized through a hardware time synchronization module, so as to guarantee the space-time consistency of multi-modal data. The data is transmitted to the cloud server in real time through a high-speed communication link and enters a detection data processing queue.

[0031] The cloud AI deep analysis and abnormal area positioning CNN model extracts features from the visible light image, and adopts a multi-scale convolution kernel to capture crack features of different widths. In a certain elevation section, the gray value abnormal area is detected, the edge presents a continuous linear feature, and it is preliminarily determined as a “potential crack area”, and the coordinate range of the area in the two-dimensional image is output. The laser radar point cloud data of the corresponding area is analyzed synchronously, it is found that the surface curvature change rate exceeds the normal threshold, and it is further confirmed that there is a geometric deformation anomaly, triggering an abnormal warning.

[0032] In an embodiment of the present application, an immediate re-inspection instruction is generated for an emergency exception (such as main tower crack propagation), and a reasonable re-inspection time window is set for a low-level exception; preset flight path information such as waypoint coordinates and shooting parameters is stored, including flight distance, no-fly zone, endurance limit and other constraint conditions, to form a structured flight path data unit; the flight path planning module supports manual editing and automatic optimization, stores waypoint parameters and constraint conditions, and meets diversified inspection requirements; a multi-dimensional decision matrix is constructed, and factors such as the importance of the structure, the severity of the problem, and the timeliness of the detection are considered to generate differentiated re-inspection instructions, and match different sensor configurations and execution strategies. Real-time acquisition of parameters such as position, power, sensor working state, and flight attitude ensures accurate assessment of equipment capabilities; multi-period detection data is integrated, re-inspection data is compared with historical records, and a structured report is generated, including an abnormal position map, a severity trend graph, and maintenance recommendations.

[0033] In an embodiment of the present application, the abnormal severity score is a dynamic threshold value: If the abnormal area is located at a key connection part of the main tower or the stay cable, the preset severity threshold value is automatically reduced; The key connection parts include the main tower, the main beam of the cable-stayed bridge, and the stay cable; when the abnormal area is located at the key connection part, the preset severity threshold value is automatically reduced by 30-50%.

[0034] In an embodiment of the present application, a mapping relationship between a two-dimensional image and a three-dimensional model is established through camera calibration parameters, potential problem areas are located to specific elevation sections of the main tower three-dimensional model, and corresponding detection waypoints are associated.

[0035] In an embodiment of the present application, the preset flight path is dynamically split into independent sub-paths, including: According to the distribution of the abnormal area, the preset flight path is split into the sub-paths covering the abnormal area and a reserved baseline path; The sub-paths are assigned priorities, and the priorities are calculated based on the abnormal severity score and the structure importance weight.

[0036] The priority calculation model is: Priority P = α × Abnormal Severity Score S + β × Structure Importance Weight I Wherein, the abnormal severity score S: the value range [0, 10] is output by the cloud AI deep recognition model, and includes crack width and rust area proportion quantitative indicators; The structure importance weight I: determined according to structural mechanics analysis, the value range [0, 1]; α and β are weight coefficients.

[0037] In a specific embodiment, the severity assessment and priority determination includes: Query the bridge detection database to confirm that the area is a first-time crack anomaly and there is no historical defect record in the surrounding area; Based on the multi-modal data, estimate the crack parameters and determine it as a medium severity anomaly.

[0038] According to the three-dimensional decision matrix, a "medium priority + re-inspection within a limited time" instruction is generated. This instruction triggers the route splitting process, while carrying sensor configuration parameters (visible light camera long focus mode, laser radar high density scanning) and flight parameters (limited flight speed, single flight point hovering time).

[0039] In a specific embodiment of the present application, the main tower key connection parts: I=0.9 (such as the connection between the tower column and the beam); cable anchor end: I=0.85 (such as cable beam anchor end); main beam midspan region: I=0.8 (such as 1 / 3 region in the middle of the span); other regions: I=0.6~0.7; α=0.7, β=0.3 (which can be trained and adjusted through historical failure data).

[0040] In an embodiment of the present application, the dynamic path planning algorithm determines the problem flight points, the distance between the problem flight points, and the priority of the sub-route based on the abnormal area to generate a key point re-inspection sub-route; According to the bridge structure segmentation rule, each detection section contains multiple detection flight points, and each detection flight point is associated with spatial coordinates, shooting angle, sensor configuration and other parameters. When an abnormal area is detected, the system automatically matches its corresponding detection section and adjacent section, expands the protection flight points upwards and downwards based on the abnormal flight point, searches for the smallest connected subgraph containing the target area, ensures that the sub-route covers the abnormal area and its adjacent key section, and eliminates irrelevant flight points to reduce invalid flight; Specifically, the graph theory algorithm is used to cut out a sub-route containing only the target area and necessary protection flight points from the original route, and irrelevant flight points are eliminated to reduce invalid flight distance.

[0041] Each flight point contains three-dimensional coordinates (x, y, z), attitude angle (yaw angle ψ, pitch angle θ) and dwell time t; according to the structure segmentation logic, the upper and lower adjacent flight points are automatically added as protection areas to avoid missing the range where the anomaly may extend; check whether the sub-route contains complete detection view angle, if there is a view angle missing, supplement the necessary flight points from the preset route to ensure the integrity of the re-inspection data; Optimize the order of the flight points in the sub-route, cluster all the flight points, and generate the local optimal path.

[0042] In a specific embodiment, the sub-route retains multiple-angle detection perspectives such as overhead, side, and overhead, to ensure all-around coverage of the defect area. For example, for a crack in a certain section of the main tower, the sub-route only contains the key waypoints of the section and the adjacent sections above and below, reducing the invalid flight distance.

[0043] After the UAV completes the current inspection task, the dynamic path planning module generates a re-inspection path in combination with the following data: the remaining power of the UAV, the flight attitude, the sensor working state, the sub-route waypoint coordinates, the priority level, the detection time window, etc. The sub-route is dynamically planned. Through the sub-route, the re-inspection work is performed, the real-time image acquisition of the bridge to be detected is completed according to the preset flight route, and the collected images are transmitted to the cloud service platform in real time. Through multi-angle and high-resolution image acquisition, the direction, length, and edge detail features of the cracks on the surface of the bridge structure are completely captured; the laser radar is simultaneously started to scan the suspected disease area in a high-density scanning mode to accurately measure the crack depth, width, and surrounding structure surface roughness; the infrared thermal imager is simultaneously scanned to analyze the heat conduction abnormal features and assist in judging the influence of the cracks on the heat transfer performance of the bridge structure. The high-definition images are subjected to crack identification and classification using a deep learning algorithm, and the crack positions are automatically labeled; the re-inspection data are combined to generate a bridge detection report based on the confirmation results: Specifically, in combination with the confirmation results, if the confirmation results are correct, that is, the marked position result is correct, and the crack exists and the crack severity is greater than the preset severity, the confirmed marked position of the crack existing and the crack severity being greater than the preset severity is retained, otherwise, the result of the re-confirmation is retained; the bridge detection report containing all crack positions and crack severities is generated based on all judgment and confirmation results.

[0044] In an embodiment of the present application, the frequency of the same position anomaly appearing in the inspection report is counted, and if the frequency exceeds a preset threshold, the position is marked as a key monitoring point, and the priority of the sub-route of the key monitoring point is automatically increased in subsequent inspections.

[0045] In another embodiment of the present application, a digital twin model of the cable-stayed bridge structure is constructed in the cloud, the re-inspection results of the UAV are synchronized in real time with the three-dimensional model data, and the abnormal area and its development trend are dynamically displayed through picture comparison.

[0046] The present application can also generate a geographic reference model: The UAV carrying a camera completes real-time image acquisition of the bridge to be detected according to the preset planar three-dimensional route of the flight route, and transmits the collected images to the cloud in real time. A three-dimensional model of the bridge is constructed using the collected high-definition images and multi-dimensional data in combination with geographic information system technology. The laser radar point cloud, infrared thermal imaging data and dense point cloud are fused, a model coordinate is optimized through a data registration algorithm, and geometric accuracy is improved. Feature points and descriptors are extracted, a corresponding relationship between images is established, and a basis for three-dimensional reconstruction is provided. An SfM technology is used, a camera relative pose is calculated through a feature corresponding relationship between images, a three-dimensional structure of a scene is restored, and a sparse point cloud containing visual features is generated. A three-dimensional point cloud model is aligned with a geographic coordinate system, historical detection data and other information are combined, and a geographic reference model with spatial positioning and attribute query functions is formed.

[0047] In one embodiment of the present application, the unmanned aerial vehicle carries the multi-sensor system, collects environmental lighting data in real time and transmits it to the cloud; the cloud generates wavelength adjustment instructions according to the environmental lighting data, and controls the light source of the unmanned aerial vehicle to emit light of a specific wave band matching the material of the cable-stayed bridge.

[0048] As shown in Figure 2 The present application also provides an unmanned aerial vehicle inspection route automatic splitting and recombination system, which realizes the method as described in any of the above, comprising: A multi-sensor data acquisition module is used to complete image and multi-dimensional data acquisition of the main tower, stay cables and bridge deck beams of the cable-stayed bridge according to a preset route, and to transmit the image and the multi-dimensional data to the cloud in real time; an anomaly detection module is used to analyze the image and the multi-dimensional data in real time, identify and mark abnormal areas of the cable-stayed bridge structure, and the marking result includes marking of abnormal position, type and severity; a route dynamic management module is used to dynamically split the preset route into independent sub-routes based on the marking result of the abnormal areas, in combination with a three-dimensional model of the cable-stayed bridge and a structure segmentation logic, to generate a review instruction containing a spatial range, an execution order and a priority; a path planning engine module is used to generate a locally optimal path according to the priority of the sub-routes, real-time state of the unmanned aerial vehicle and environmental data, using a dynamic path planning algorithm, to control the unmanned aerial vehicle to perform a re-inspection task of the sub-routes and generate a re-inspection result. A report generation module is used to generate a cable-stayed bridge structure inspection report in combination with the re-inspection result, and to update an abnormal area database.

[0049] A specific implementation of the method and system of the present application is as follows: The unmanned aerial vehicle carries a multi-sensor system including a high-definition camera, a laser radar, an infrared camera and a hyperspectral sensor, and collects data of the cable-stayed bridge and the bridge deck according to a preset flight route. The high-definition camera obtains a sequence of images of the whole surface of the bridge at a preset overlap rate and flight height. The sensors collect high-definition images and multi-dimensional data of the slope of the highway bridge in real time, including information of topography, vegetation coverage, geological structure and temperature change. The multi-dimensional data such as laser radar data, infrared data and elevation data provide accurate spatial information for the construction of a three-dimensional model.

[0050] The data collected by the unmanned aerial vehicle in flight is transmitted back to a cloud platform, and image registration technology is used to align the sequenced high-definition images to a common reference frame.

[0051] Feature points and feature descriptors are extracted from the aligned image set through computer vision technology. The specific steps are as follows: the ORB algorithm is used to detect corner points, edges and other feature points in each image to generate binary descriptors; the correspondence relationship between feature points is established based on the similarity of the descriptors to form a cross-image feature point trajectory, which provides key corresponding information for subsequent three-dimensional structure recovery.

[0052] The structure from motion (SfM) algorithm is used to calculate the image relative pose and scene three-dimensional structure based on the feature point correspondence relationship to generate a sparse three-dimensional point cloud model. The specific implementation steps of the SfM algorithm are as follows: An initial scene (the image with the highest overlap degree) is selected, the essential matrix of the camera is estimated by the eight-point method, the relative rotation matrix (R) and the translation vector (T) are calculated, and the first batch of three-dimensional coordinate points are generated by triangulation; The camera parameters and three-dimensional point coordinates are optimized by Bundle Adjustment; The remaining images are added one by one, the PnP algorithm is used to estimate the pose of the new image, the three-dimensional point cloud is expanded by triangulation, and local Bundle Adjustment optimization is performed simultaneously; after all the images are processed, global Bundle Adjustment is performed to generate a three-dimensional scene model of the bridge containing the camera pose and sparse points.

[0053] The multi-view stereo matching (MVS) technology is used to densify the sparse point cloud; The LiDAR point cloud and the MVS dense point cloud are fused to optimize the accuracy and integrity of the three-dimensional model; In the geographic information system (GIS), the three-dimensional model is aligned with the geographic coordinate system. The vertex coordinates of the model are converted into latitude, longitude and elevation; The model is given structured attribute information to form a georeferenced three-dimensional model of the bridge, which supports the spatial positioning needs of subsequent analysis; The generated three-dimensional model is verified for accuracy by comparing it with known geographic control points or historical data, resulting in a verification result that ensures the accuracy and precision of the model.

[0054] Based on the verification result, the three-dimensional model is adjusted and optimized to correct deviations, resulting in the final bridge three-dimensional model. This involves fine-tuning and optimization of the model to ensure optimal performance and applicability.

[0055] The above is a further detailed description of the present application in combination with specific / preferred embodiments, and cannot be considered as limiting the specific implementation of the present application to these descriptions. For ordinary skilled persons in the technical field to which the present application belongs, they can make several substitutions or modifications to the described embodiments without departing from the concept of the present application, and these substitutions or modifications should be considered as falling within the protection scope of the present application. In the description of the present application, the description of the terms "an embodiment", "some embodiments", "a preferred embodiment", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in combination with the embodiment or example are included in at least one embodiment or example of the present application. The illustrative description of the above terms in the present application does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In the case of no mutual contradiction, the skilled person in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples. Although the embodiments of the present application and their advantages have been described in detail, it should be understood that various changes, substitutions and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A method for automatically splitting and reassembling a drone inspection route, characterized in that: Based on cable-stayed bridge structural members; Using a drone equipped with a multi-sensor system, the images and multi-dimensional data of the main tower, stay cables and bridge deck beams of the cable-stayed bridge are collected according to a preset route, and the images and multi-dimensional data are transmitted to the cloud in real time; The cloud performs real-time analysis on the image and the multi-dimensional data, identifies and marks abnormal areas of the cable-stayed bridge structure, and the marking results include the marked abnormal location, type, and severity; Based on the marking results of the abnormal area, combined with the three-dimensional model of the cable-stayed bridge and the structural component segmentation logic, the preset route is dynamically split into independent sub-routes, and a review instruction including the spatial range, execution order and priority is generated; Using a dynamic path planning algorithm, a local optimal path is generated according to the priority of the sub-route, the real-time status of the drone, and environmental data, and the drone is controlled to perform the re-inspection task of the sub-route and generate a re-inspection result; An inspection report of cable-stayed bridge structural components is generated based on the re-inspection results, and an abnormal area database is updated.

2. The method for automatically splitting and reassembling the inspection route of a UAV according to claim 1, characterized in that: The generation of the preset route includes the following steps: Based on the three-dimensional model of the cable-stayed bridge, the main tower of the cable-stayed bridge, the stay cables and the bridge deck are divided into a plurality of key inspection areas; According to the geometric features and detection requirements of each key detection area, the preset route covering the entire bridge is generated, and the detection parameters of each key detection area are predefined, including flight altitude, shooting angle and sensor configuration.

3. The method for automatically splitting and reassembling the inspection route of a UAV according to claim 2, characterized in that: Performing real-time analysis of the image and the multi-dimensional data using a cloud-based AI deep recognition model, specifically including: The multi-branch first CNN model employed by the cloud-based AI deep recognition model adaptively selects a matching branch based on the collected image to determine whether a potential crack exists in the current view. If so, the location of the potential crack is marked in the view along with a preset color and format. Each branch in the first CNN model is trained and determines whether a potential crack exists in the cable-stayed bridge. Using the multi-branch second CNN model employed by the cloud-based AI deep recognition model, the branch that adaptively matches the color and format in the view determines whether cracks exist at the marked location, as well as the location and severity of the cracks. Each branch of the second CNN model is trained and determines whether cracks exist at the marked location in the image, as well as the location and severity of the cracks. The input of the cloud-based AI deep recognition model is a fusion of the image and the lidar point cloud data, and the lidar point cloud data is obtained by the multi-sensor; the output includes the abnormality location coordinates, the abnormality type label and the abnormality severity score.

4. The method for automatically splitting and reassembling the inspection route of a UAV according to claim 3, characterized in that: The abnormality severity score is a dynamic threshold: If the abnormal area is located at a key connection point of the main tower or the inclined cable, the preset severity threshold is automatically lowered; The key connection parts include the main tower, main beam and the stay cable of the cable-stayed bridge; When the abnormal area is located at the critical connection part, the preset severity threshold is automatically reduced by 30%-50%.

5. The method for automatically splitting and reassembling the inspection route of a UAV according to claim 4, characterized in that: Dynamically splitting the preset route into independent sub-routes includes: According to the distribution of the abnormal area, the preset route is split into the sub-routes covering the abnormal area and the retained baseline route; Assign a priority to the sub-route, wherein the priority is calculated based on the weighted calculation of the severity score of the anomaly and the importance of the structural component. The priority calculation model is: Priority P = α × abnormal severity score S + β × structural component importance weight I Among them, the abnormality severity score S: the value range is [0,10], which is output by the cloud AI deep recognition model and includes quantitative indicators such as crack width and rust area ratio; Importance weight of structural components I: determined according to structural mechanics analysis, with a value range of [0,1]; α and β are weight coefficients.

6. The method for automatically splitting and reassembling the inspection route of a UAV according to claim 5, characterized in that: The dynamic path planning algorithm determines the problem waypoints, the distances between the problem waypoints, and the priorities of the sub-routes through the abnormal area, and generates sub-routes for key point rechecks; Taking the abnormal waypoint as the core, search for the minimum connected subgraph containing the target area, ensure that the sub-route covers the abnormal area and its adjacent key sections, and eliminate irrelevant waypoints; Each of the waypoints includes three-dimensional coordinates, attitude angles and dwell time t; Automatically add upper and lower adjacent waypoints as protection areas based on the structural component segmentation logic; Check whether the sub-route contains a complete detection perspective, and if there is a missing perspective, add necessary waypoints from the preset route; The order of waypoints within the sub-route is optimized, all the waypoints are clustered, and the local optimal path is generated.

7. The method for automatically splitting and reassembling a drone inspection route according to claim 1, characterized in that: Also includes: The drone is equipped with the multi-sensor system to collect ambient light data in real time and transmit the data to the cloud; The cloud generates a wavelength adjustment instruction based on the ambient light data to control the light source of the drone to emit light of a specific wavelength band that matches the material of the cable-stayed bridge.

8. The method for automatically splitting and reassembling a drone inspection route according to claim 1, wherein: The frequency of abnormal occurrences at the same location in the inspection report is counted. If the frequency exceeds a preset threshold, the location is marked as a key monitoring point, and the sub-route priority of the key monitoring point is automatically increased in subsequent inspections.

9. The method for automatically splitting and reassembling a drone inspection route according to claim 1, characterized in that: Also includes: A digital twin model of the cable-stayed bridge structure is constructed in the cloud, the re-inspection results of the drone and the three-dimensional model data are synchronized in real time, and abnormal areas and their development trends are dynamically displayed through image comparison.

10. A system for automatically splitting and reassembling drone inspection routes, characterized in that: Implementing the method according to any one of claims 1 to 9, comprising: A multi-sensor data acquisition module is used to complete the image and multi-dimensional data collection of the main tower, stay cables and bridge deck beams of the cable-stayed bridge according to the preset route, and transmit the images and multi-dimensional data to the cloud in real time; an anomaly detection module, configured to perform real-time analysis on the image and the multi-dimensional data, identify and mark abnormal areas of the cable-stayed bridge structure, and include the marked abnormal location, type, and severity; A route dynamic management module is used to dynamically split the preset route into independent sub-routes based on the marking results of the abnormal area, combined with the three-dimensional model of the cable-stayed bridge and the structural component segmentation logic, and generate a review instruction including the spatial scope, execution order and priority; A path planning engine module is used to generate a local optimal path using a dynamic path planning algorithm based on the priority of the sub-route, the real-time status of the drone, and environmental data, and control the drone to perform the re-inspection task of the sub-route and generate a re-inspection result; The report generation module is used to generate a cable-stayed bridge structural member inspection report based on the re-inspection results and update the abnormal area database.

Images