Bridge inspection unmanned aerial vehicle route planning method and system

Abstract

The application provides a bridge inspection unmanned aerial vehicle route planning method and system, and belongs to the technical field of bridge inspection. The method comprises the following steps: before the unmanned aerial vehicle takes off, an illumination reference model is established and normalized according to the three-dimensional point cloud of the bridge structure, low-illumination light distribution, material and spatial layout; in flight, light field data are collected by using a multispectral camera and an illumination sensor, and a dynamic light field mapping is generated by comparing the difference with the reference model; at the same time, obstacle point clouds are collected by using a laser radar and a millimeter wave radar; the dynamic light field mapping and the point clouds are input into a time series prediction model to predict the obstacle probability and position trend; if the probability is higher than a threshold value, roll, pitch and speed adjustment parameters are generated and fine adjustment is performed; when the probability is reduced and the illumination is improved, the original planned route is switched back. The method improves the obstacle avoidance performance of the unmanned aerial vehicle under low illumination, realizes stable environment perception regardless of illumination, and is beneficial to high-precision inspection and structure health data collection.

Description

Technical Field

[0001] This invention relates to the field of bridge inspection technology, and in particular to a method and system for planning the flight path of a bridge inspection drone. Background Technology

[0002] Currently, bridge inspection drones mostly use GNSS navigation, combined with sensors such as visible light cameras, lidar or millimeter-wave radar, to complete three-dimensional reconstruction of the structure, route planning and obstacle avoidance through point cloud data. However, the existing technology still has certain defects in inspection conditions such as low light, denied environment, complex spatial structure and multiple materials under the bridge: (1) Visual obstacle avoidance is highly dependent on ambient light. Under low light, the image signal-to-noise ratio drops sharply, the accuracy of obstacle recognition and spatial ranging is greatly reduced, and the risk of drone collision increases significantly; (2) Single sensors have inherent performance shortcomings. Existing multi-sensor fusion schemes mostly use fixed weights to simply superimpose data, which does not adapt to the differences in reflection characteristics of different materials such as steel and concrete, resulting in poor consistency of perception data and insufficient environmental adaptability; (3) The obstacle avoidance logic is mainly based on post-event passive response and lacks a joint prediction mechanism for the dynamic changes of ambient light field and obstacle trends. The obstacle avoidance action is prone to cause sudden changes in the flight path, affecting flight stability and the quality of inspection data collection.

[0003] Therefore, existing bridge inspection drones need to be improved to meet the needs of safe and efficient autonomous inspection in complex environments with no signal and low light conditions under bridges. Summary of the Invention

[0004] To overcome the problems existing in related technologies, one of the objectives of this invention is to provide a method for planning the flight path of a bridge inspection drone. This method can improve the obstacle avoidance performance of the bridge inspection drone in low-light environments, achieve stable environmental perception independent of lighting conditions, and facilitate high-precision inspection and structural health data collection by the drone.

[0005] A method for planning the flight path of a bridge inspection drone includes: S1. Before the drone takes off, based on the three-dimensional point cloud data of different structural locations of the bridge, low-light illumination distribution data, material type and spatial layout labels, establish an illumination reference model, and normalize the illumination reference model to generate a normalized illumination reference matrix. S2. During flight, real-time ambient light field data at different locations under the bridge are collected using a multispectral camera and a light sensor to form a multi-location light field sampling sequence. S3. Perform difference calculation between the multi-location light field sampling sequence and the normalized illumination reference matrix to generate dynamic light field mapping data; S4. Use lidar and millimeter-wave radar to collect real-time obstacle point cloud data at different structural locations under the bridge to form a multi-location obstacle point cloud dataset. S5. Simultaneously input the dynamic light field mapping data and the multi-location obstacle point cloud dataset into the temporal obstacle prediction model to generate prediction results of the probability of obstacle appearance and the trend of position change on the future flight path. S6. When the probability of an obstacle appearing in the prediction result is higher than the preset threshold, generate a flight attitude adjustment parameter group that includes roll, pitch and speed adjustment, and execute flight attitude fine-tuning control. S7. When the probability of obstacles appears decreases and the light field conditions improve, switch to the original planned route parameter group corresponding to the normalized illumination reference matrix.

[0006] In a preferred embodiment of the present invention, after step S7, the method further includes: S8. Record each flight path attitude adjustment and corresponding environmental data, and update the parameters of the timing obstacle prediction model.

[0007] In a preferred embodiment of the present invention, step S1 includes: generating a structural position index matrix based on three-dimensional point cloud data; Material type acquisition is performed on the structural location index matrix to obtain the material spectral characteristic parameter set and bind it to the corresponding structural location index; The low-illuminance light distribution data is registered with the structural location index matrix and the material spectral characteristic parameter set to form a three-dimensional mapping table of location-material-illuminance. Spatial layout labels are generated for the 3D mapping table, and spatial topology analysis is used to calculate the relative height, span, and proximity of each structure location. The input condition matrix for the illumination reference model is generated based on the 3D mapping table and spatial layout labels. The material type acquisition includes: using a spectral feature separation algorithm to perform band decomposition and feature extraction on the original spectral signal to obtain the material feature vector; and using a material spectral characteristic parameter calculation algorithm to map the material feature vector into a set of material spectral characteristic parameters including reflectance coefficient, absorption coefficient, peak position, and peak width.

[0008] In a preferred embodiment of the present invention, step S1, the normalization process includes: Calculate the normalized coefficient matrix of material reflectance based on the reflectance coefficient in the set of material spectral characteristic parameters; The normalized coefficient matrix is ​​multiplied point-by-point with the input condition matrix of the illumination reference model to obtain the reflectivity-corrected illumination matrix. The position illumination mean is normalized by normalizing the reflectivity-corrected illumination matrix to obtain the position brightness normalized matrix; The position brightness normalization matrix is ​​combined with the spatial layout labels to perform spatial topology weighted average smoothing to obtain a smoothed illumination matrix. The smoothed illumination matrix is ​​normalized by global maximum and minimum values ​​to generate a normalized illumination reference matrix.

[0009] In a preferred embodiment of the present invention, step 2 includes: The light field acquisition task is planned based on the structure location index in the normalized illumination reference matrix, and the real-time multi-band illumination signal matrix of each structure location is obtained using a multispectral camera. The multi-band illumination signal matrix is ​​spectrally decomposed into single-band light intensity vectors; radiometric correction is performed on the single-band light intensity vectors to eliminate the effects of sensor system noise and temperature drift. The radiation-corrected light intensity data is bound to the structure location index with location tags to form a location-bound illumination matrix; The location-bound illumination matrix is ​​continuously sorted according to the flight timestamp to generate a multi-location light field sampling time sequence matrix.

[0010] In a preferred embodiment of the present invention, step S3 includes: A position-by-position difference processing is performed on the multi-position light field sampling time sequence matrix and the normalized illumination reference matrix to obtain the brightness difference matrix; Calculate the statistical characteristics of the differences by grouping according to structural location, and generate a location difference feature matrix; Extract the temporal change slope and fluctuation amplitude from the location difference feature matrix to generate a temporal difference trend matrix; By combining spatial layout labels, a spatial topological weighted fusion of the temporal difference trend matrix is ​​performed to generate a spatial trend fusion difference matrix. Dynamic light field interpolation is performed on the spatial trend fusion difference matrix to form a dynamic light field mapping data matrix.

[0011] In a preferred embodiment of the present invention, step S4 includes: The location of the structure under the bridge was scanned using lidar to obtain the lidar point cloud matrix of the structure location; Extract the reflection intensity features of the lidar point cloud matrix to form a reflection intensity feature matrix; Millimeter-wave radar is used to scan the same structural location to obtain the penetration characteristic parameter matrix, and multi-source point cloud feature fusion is performed with the reflection intensity feature matrix to generate the obstacle multi-source feature fusion matrix. Three-dimensional spatial filtering is performed on the multi-source feature fusion matrix to extract the obstacle boundary feature matrix; The boundary feature matrix and the structural location index are bound to location labels and grouped to form a multi-location obstacle point cloud dataset.

[0012] In a preferred embodiment of the present invention, step S5 includes: The dynamic light field mapping data and the multi-location obstacle point cloud dataset are time-stamped and aligned to generate a time-aligned multi-source input matrix. The light field difference features and obstacle boundary structure features in the time-aligned multi-source input matrix are converted into a unified feature vector space to form a multi-source unified feature matrix. Temporal convolutional fusion is performed on the multi-source unified feature matrix to generate a location-dynamically fused feature matrix; By combining the relative height, span, and proximity relationships between structural locations, the location dynamic fusion feature matrix is ​​spatially topologically weighted to generate a spatial relationship weighted fusion feature matrix; The spatial relationship weighted fusion feature matrix is ​​standardized and a fixed-length temporal input segment is generated, which is then input into the temporal obstacle prediction model.

[0013] In a preferred embodiment of the present invention, step S6 includes: The probability values ​​of obstacle occurrence are extracted from the prediction results at each position to form an obstacle probability value matrix; The probability value matrix is ​​compared with a preset safety threshold to generate a judgment mark matrix; The judgment mark matrix is ​​registered with the position change trend matrix to generate the trigger position dynamic offset matrix; Based on the dynamic offset matrix of the trigger position and the reference parameter matrix of the flight attitude, the roll adjustment, pitch adjustment and velocity adjustment are calculated position by position to generate a position and attitude adjustment parameter matrix; the adjustment parameters are then grouped and summarized according to structural position to generate a grouped attitude adjustment parameter table. Step S7 includes: Extract the obstacle occurrence probability matrix based on the prediction results for each time slice; By trend correlation and fusion of the probability matrix and dynamic light field mapping data, a joint trend matrix of light field probability is generated. The joint trend matrix is ​​subjected to a dual-condition threshold determination of obstacle occurrence probability and illumination improvement coefficient to generate a route switching trigger marker matrix; The trigger marker matrix is ​​mapped to the original planned route parameter matrix to generate the route recovery position parameter matrix; The original planned route parameter set is converted into a switching command and loaded into the flight control cache to execute the route switch; The difference between the switched flight attitude and the original planned route parameters is analyzed. If the stability is lower than the threshold, a secondary attitude fine-tuning is triggered.

[0014] The second objective of this invention is to provide a bridge inspection drone route planning system, which is used to implement the bridge inspection drone route planning method described above.

[0015] The beneficial effects of this invention are as follows: This invention provides a method and system for planning the flight path of a bridge inspection UAV. The method employs a multi-source sensor fusion architecture combining a multispectral camera, an illumination sensor, lidar, and millimeter-wave radar, making obstacle avoidance capabilities independent of visible light illuminance. The millimeter-wave radar compensates for the significant performance degradation of a single visual sensor when brightness decreases. Through multi-band fusion and collaborative perception by multiple radar sources, the obstacle detection accuracy of the UAV can be improved. This method establishes an illumination reference model and normalizes the illumination distribution at different structural locations, eliminating the influence of material reflectivity differences on light field mapping. Based on this, the method performs difference calculations between the real-time multi-location light field sampling sequences and the normalized illumination reference matrix to generate dynamic light field mapping data. This enables the system to stably capture changes in the ambient light field even under low-light conditions, achieving stable environmental perception independent of illumination conditions. The method also incorporates a complete flight path recovery mechanism: when the probability of an obstacle appearing decreases and the light field conditions improve, the system automatically switches to the original planned flight path parameter group corresponding to the normalized illumination reference matrix, ensuring seamless switching between local obstacle avoidance actions and the original planned flight path. Furthermore, this method loads 3D point cloud data, low-light illumination distribution data, material type, and spatial layout labels for different structural locations of the bridge before takeoff to establish an illumination baseline model. During flight, real-time point cloud data is collected using lidar and millimeter-wave radar, and combined with location labels to form a multi-location obstacle point cloud dataset. The establishment of 3D mapping between material properties, light field, and obstacle spatial locations enables the model to have accurate spatial registration capabilities, reducing positioning errors and facilitating high-precision inspection and structural health data collection. Attached Figure Description

[0016] Figure 1 This is a flowchart of a bridge inspection drone route planning method provided in an embodiment of the present invention. Detailed Implementation

[0017] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0018] Currently, most inspection drones use GNSS navigation, coupled with sensors such as visible light cameras, lidar, or millimeter-wave radar, to complete structural 3D reconstruction, flight path planning, and obstacle avoidance through point cloud data. However, existing technologies still have certain shortcomings in inspection conditions such as low illumination under bridges, denied environments, complex spatial structures, and multiple materials: (1) Visual obstacle avoidance is highly dependent on ambient light. Under low illumination, the image signal-to-noise ratio drops sharply, the accuracy of obstacle recognition and spatial ranging is greatly reduced, and the risk of drone collision increases significantly; (2) Single sensors have inherent performance shortcomings. Existing multi-sensor fusion schemes mostly use fixed weights to simply superimpose data, which does not adapt to the differences in reflection characteristics of different materials such as steel and concrete, resulting in poor consistency of perception data and insufficient environmental adaptability; (3) The obstacle avoidance logic is mainly based on post-event passive response and lacks a joint prediction mechanism for dynamic changes in the ambient light field and obstacle trends. Obstacle avoidance actions are prone to cause sudden changes in the flight path, affecting flight stability and the quality of inspection data collection.

[0019] Therefore, this application provides a method for planning the flight path of a bridge inspection drone to overcome the shortcomings of the prior art.

[0020] Example 1 like Figure 1 As shown in the figure, this embodiment provides a method for planning the flight path of a bridge inspection drone, including: S1. Before the drone takes off, based on the three-dimensional point cloud data of different structural locations of the bridge, low-light illumination distribution data, material type and spatial layout labels, establish an illumination reference model, and normalize the illumination reference model to generate a normalized illumination reference matrix.

[0021] Specifically, step S1 includes: Generate a structural location index matrix based on 3D point cloud data; Material type acquisition is performed on the structural location index matrix to obtain the material spectral characteristic parameter set and bind it to the corresponding structural location index; The low-illuminance light distribution data is registered with the structural location index matrix and the material spectral characteristic parameter set to form a three-dimensional mapping table of location-material-illuminance. Spatial layout labels are generated for the 3D mapping table, and spatial topology analysis is used to calculate the relative height, span, and proximity of each structure location. The input condition matrix for the illumination reference model is generated based on the 3D mapping table and spatial layout labels. The material type acquisition includes: using a spectral feature separation algorithm to perform band decomposition and feature extraction on the original spectral signal to obtain the material feature vector; and using a material spectral characteristic parameter calculation algorithm to map the material feature vector into a set of material spectral characteristic parameters including reflectance coefficient, absorption coefficient, peak position, and peak width.

[0022] More specifically, in step S1, the normalization process includes: Calculate the normalized coefficient matrix of material reflectance based on the reflectance coefficient in the set of material spectral characteristic parameters; The normalized coefficient matrix is ​​multiplied point-by-point with the input condition matrix of the illumination reference model to obtain the reflectivity-corrected illumination matrix. The position illumination mean is normalized by normalizing the reflectivity-corrected illumination matrix to obtain the position brightness normalized matrix; The position brightness normalization matrix is ​​combined with the spatial layout labels to perform spatial topology weighted average smoothing to obtain a smoothed illumination matrix. The smoothed illumination matrix is ​​normalized by global maximum and minimum values ​​to generate a normalized illumination reference matrix.

[0023] S2. During flight, real-time ambient light field data at different locations under the bridge are collected using a multispectral camera and a light sensor to form a multi-location light field sampling sequence; the spectral bands collected by the multispectral camera include at least the visible light band, the near-infrared band, and the short-wave infrared band.

[0024] Further, step 2 includes: The light field acquisition task is planned based on the structure location index in the normalized illumination reference matrix, and the real-time multi-band illumination signal matrix of each structure location is obtained using a multispectral camera. The multi-band illumination signal matrix is ​​spectrally decomposed into single-band light intensity vectors; radiometric correction is performed on the single-band light intensity vectors to eliminate the effects of sensor system noise and temperature drift. The radiation-corrected light intensity data is bound to the structure location index with location tags to form a location-bound illumination matrix; The location-bound illumination matrix is ​​continuously sorted according to the flight timestamp to generate a multi-location light field sampling time sequence matrix.

[0025] S3. Perform difference calculation between the multi-position light field sampling sequence and the normalized illumination reference matrix to generate dynamic light field mapping data.

[0026] Further, step S3 includes: A position-by-position difference processing is performed on the multi-position light field sampling time sequence matrix and the normalized illumination reference matrix to obtain the brightness difference matrix; Calculate the statistical characteristics of the differences by grouping according to structural location, and generate a location difference feature matrix; Extract the temporal change slope and fluctuation amplitude from the location difference feature matrix to generate a temporal difference trend matrix; By combining spatial layout labels, a spatial topological weighted fusion of the temporal difference trend matrix is ​​performed to generate a spatial trend fusion difference matrix. Dynamic light field interpolation is performed on the spatial trend fusion difference matrix to form a dynamic light field mapping data matrix.

[0027] S4. Use lidar and millimeter-wave radar to collect real-time obstacle point cloud data at different structural locations under the bridge, forming a multi-location obstacle point cloud dataset.

[0028] Further, step S4 includes: The location of the structure under the bridge was scanned using lidar to obtain the lidar point cloud matrix of the structure location; Extract the reflection intensity features of the lidar point cloud matrix to form a reflection intensity feature matrix; Millimeter-wave radar is used to scan the same structural location to obtain the penetration characteristic parameter matrix, and multi-source point cloud feature fusion is performed with the reflection intensity feature matrix to generate the obstacle multi-source feature fusion matrix. Three-dimensional spatial filtering is performed on the multi-source feature fusion matrix to extract the obstacle boundary feature matrix; The boundary feature matrix and the structural location index are bound to location labels and grouped to form a multi-location obstacle point cloud dataset.

[0029] S5. Simultaneously input the dynamic light field mapping data and the multi-location obstacle point cloud dataset into the temporal obstacle prediction model to generate prediction results of the probability of obstacle appearance and the trend of position change on the future flight path.

[0030] Further, step S5 includes: The dynamic light field mapping data and the multi-location obstacle point cloud dataset are time-stamped and aligned to generate a time-aligned multi-source input matrix. The light field difference features and obstacle boundary structure features in the time-aligned multi-source input matrix are converted into a unified feature vector space to form a multi-source unified feature matrix. Temporal convolutional fusion is performed on the multi-source unified feature matrix to generate a location-dynamically fused feature matrix; By combining the relative height, span, and proximity relationships between structural locations, the location dynamic fusion feature matrix is ​​spatially topologically weighted to generate a spatial relationship weighted fusion feature matrix; The spatial relationship weighted fusion feature matrix is ​​standardized and a fixed-length temporal input segment is generated, which is then input into the temporal obstacle prediction model.

[0031] S6. When the probability of an obstacle appearing in the prediction result is higher than the preset threshold, generate a flight attitude adjustment parameter group that includes roll, pitch and speed adjustment, and execute flight attitude fine-tuning control.

[0032] Further, step S6 includes: The probability values ​​of obstacle occurrence are extracted from the prediction results at each position to form an obstacle probability value matrix; The probability value matrix is ​​compared with a preset safety threshold to generate a judgment mark matrix; the judgment mark matrix is ​​registered with the position change trend matrix to generate a trigger position dynamic offset matrix. Based on the dynamic offset matrix of the trigger position and the reference parameter matrix of the flight attitude, the roll adjustment, pitch adjustment and velocity adjustment are calculated position by position to generate a position and attitude adjustment parameter matrix; the adjustment parameters are then grouped and summarized according to structural position to generate a grouped attitude adjustment parameter table. More specifically, the execution of flight attitude fine-tuning control includes: Map the flight path attitude adjustment parameter group to the flight control hardware driver interface to generate an attitude adjustment task queue; The roll, pitch, and speed adjustments are written into the flight control system cache to generate an attitude control signal sequence. The attitude control signal sequence is input into the flight control execution unit to perform synchronous fine-tuning of each attitude axis of the UAV. The fine-tuning execution result is matched with the bridge structure spatial layout label with a safe distance to generate a safe distance judgment matrix. If it is qualified, the current flight path is maintained; if it is not qualified, a second attitude fine-tuning is triggered. The roll, pitch, and velocity parameters are stabilized and corrected to generate the attitude adjustment reference for the next moment.

[0033] S7. When the probability of obstacles appears decreases and the light field conditions improve, switch to the original planned route parameter group corresponding to the normalized illumination reference matrix.

[0034] Step S7 includes: Extract the obstacle occurrence probability matrix based on the prediction results for each time slice; By trend correlation and fusion of the probability matrix and dynamic light field mapping data, a joint trend matrix of light field probability is generated. The joint trend matrix is ​​subjected to a dual-condition threshold determination of obstacle occurrence probability and illumination improvement coefficient to generate a route switching trigger marker matrix; The trigger marker matrix is ​​mapped to the original planned route parameter matrix to generate the route recovery position parameter matrix; The original planned route parameter set is converted into a switching command and loaded into the flight control cache to execute the route switch; The difference between the switched flight attitude and the original planned route parameters is analyzed. If the stability is lower than the threshold, a secondary attitude fine-tuning is triggered.

[0035] Specifically, after step S7, the following is also included: S8. Record each flight path attitude adjustment and corresponding environmental data, and update the parameters of the timing obstacle prediction model.

[0036] Step S8 specifically includes: The attitude adjustment events are timestamped and associated with the corresponding environmental change data to generate an attitude-environment synchronization record matrix. The roll, pitch, speed adjustment, light field difference features, and obstacle position features are extracted from the synchronous recording matrix and converted into a model training feature vector matrix. The feature vector matrix is ​​normalized to generate a standardized feature matrix; A sliding window is used to segment continuously recorded data into fixed-length time-series samples to generate a time-series training sample matrix. The time-series training sample matrix is ​​input into the time-series obstacle prediction model, and the model weights and time-series feature mapping coefficients are iteratively optimized using an incremental learning algorithm to generate an updated model parameter set. The updated model is evaluated for prediction accuracy and environmental adaptability, and performance evaluation results are generated and archived.

[0037] This application uses the inspection of the bottom of a box girder of a cross-river bridge as an example to illustrate the method of protection sought in this application. The area has no GNSS signals, illuminance below 10 lx, and involves three materials: steel, concrete, and anti-corrosion coating.

[0038] Step S1: Establish and normalize the illumination baseline model. Generate structural position index matrix High-precision 3D point cloud data containing the structural locations of the bridge was acquired, and partitioning mapping was performed based on the point cloud spatial coordinate system. The inspection area at the bottom of the bridge was divided into 36 structural location units (6 rows × 6 columns) according to spatial coordinates. Each unit covers a spatial range of 2m × 2m, and a structural location index matrix ID_map was generated. Each index corresponds to a unique spatial coordinate interval, providing a unified spatial reference for subsequent illumination data association and positioning.

[0039] Material type collection Material type acquisition is performed on the structure location index matrix. Material spectral analysis algorithms are used to analyze the surface reflectance spectral characteristics of the point cloud, specifically including: (1) The original spectral signal matrix is ​​decomposed and its features are extracted using a spectral feature separation algorithm. The band splitting strategy Band_split divides the spectral bands into 400-700nm (visible light), 700-1100nm (near infrared) and 1100-1700nm (shortwave infrared). The feature extraction window length Win_len is set to 128 points and the signal-to-noise ratio threshold SNR_threshold is set to 25dB. The reflectivity, absorptivity and peak-valley positions of different bands are encapsulated into material feature vectors.

[0040] (2) A material spectral characteristic parameter calculation algorithm is used to map the feature vector to the material spectral characteristic parameter set. In this embodiment, the reflection peak λ_peak of steel is 520nm, the spectral width Δλ is 12nm, and the contrast ratio Contrast_ratio is 1.8; the reflection peak λ_peak of concrete is 610nm, Δλ is 20nm, and the contrast ratio Contrast_ratio is 1.2; the peak λ_peak of coating material is 450nm, Δλ is 15nm, and the contrast ratio Contrast_ratio is 2.0. A material spectral characteristic parameter set containing core parameters such as reflection coefficient, absorption coefficient, peak position, and peak width is formed.

[0041] (3) The material spectral characteristic parameter set is bound to the structural position index one by one through the structural position index binding algorithm to obtain the position-material spectral parameter index matrix. After testing, the spectral recognition accuracy reached 98.6%, and the material classification accuracy reached over 97%.

[0042] Illumination data registration Load low-light illumination distribution data at different structural locations of the bridge, perform illumination data registration based on the structural location index matrix and material spectral characteristic parameter set, and form a three-dimensional mapping table of location-material-illumination.

[0043] Spatial layout tag generation Spatial layout labels are generated for the location-material-lighting 3D mapping table. The spatial topology analysis algorithm is used to calculate the relative height, span and proximity of each structure location, and the layout labels are added to the mapping table.

[0044] Generate the input condition matrix for the illumination baseline model Based on the location-material-lighting 3D mapping table and spatial layout labels, a lighting reference model input condition matrix is ​​generated using a lighting reference model initialization algorithm.

[0045] Normalization (1) Based on the reflectance coefficient in the set of material spectral characteristic parameters, the normalized coefficient matrix of material reflectance is obtained by using the illumination normalization coefficient calculation algorithm.

[0046] (2) Perform point-by-point multiplication of the material reflectivity normalization coefficient matrix and the illumination reference model input condition matrix to obtain the reflectivity correction illumination matrix, thereby eliminating the influence of material reflectivity differences on light field mapping.

[0047] (3) Based on the reflectivity correction illumination matrix, the matrix data is grouped and statistically analyzed using the position label index and the mean normalization algorithm is used to adjust the illumination values ​​of each structure position to a uniform relative brightness range, thereby obtaining the position brightness normalization matrix.

[0048] (4) The position brightness normalization matrix and the spatial layout label matrix are subjected to illumination distribution smoothing. The spatial topology weighted average algorithm is used to realize the continuous adjustment of illumination values ​​between adjacent structural positions, and a smoothed illumination matrix is ​​generated.

[0049] (5) The maximum and minimum value normalization algorithm is used to scale all illumination data to the [0,1] normalized interval to generate a normalized illumination reference matrix.

[0050] In step S1, by establishing the structural location index matrix and accurately acquiring the spectral characteristic parameters of the materials, the precise fusion of material information and spatial location is achieved, with a spectral recognition accuracy of 98.6%, providing highly consistent input conditions for subsequent illumination data registration and benchmark model construction. Normalization eliminates the influence of differences in reflectivity among various materials such as steel, concrete, and coatings on the light field mapping, making the illumination data of structural locations of different materials comparable and providing a unified reference benchmark for dynamic light field mapping.

[0051] Step S2: Acquire real-time ambient light field data Light field acquisition task planning Based on the structural location index in the normalized illumination reference matrix, a flight path light field acquisition task was planned for the space under the bridge. Using an airborne multispectral camera, a multi-band acquisition mode (visible band 400–700 nm, near-infrared band 700–1100 nm, and short-wave infrared band 1100–1700 nm) was set to acquire the real-time multi-band illumination signal matrix for each structural location.

[0052] Spectral decomposition Spectral decomposition processing is performed on the real-time multi-band illumination signal matrix. A spectral feature separation algorithm is used to decompose the illumination signal of each band into single-band intensity vectors. During the single-band signal extraction stage, the band indices ID 1, 2, and 3 correspond to three spectral ranges, and the sampling resolution λ_res is set to 1 nm. In the intensity feature matrix, the maximum intensity value I_max in the visible light band is 2300 lx, in the near-infrared band it is 1500 lx, and in the short-wave infrared band it is 1100 lx; ​​the minimum intensity values ​​I_min are 300 lx, 200 lx, and 150 lx, respectively; and the average intensity values ​​I_avg are 1500 lx, 920 lx, and 660 lx, respectively. A noise feature filtering algorithm is used, with the low-frequency cutoff threshold f_low set to 0.5 Hz and the high-frequency cutoff threshold f_high set to 10 Hz, to filter out environmental flicker and high-frequency interference.

[0053] Radiation correction Radiometric correction processing is performed based on a single-band light intensity vector. Utilizing a standard radiometric reference coefficient provided by a high-sensitivity light sensor, a radiometric correction algorithm is employed to eliminate the influence of sensor system noise and temperature drift on light intensity data, generating a radiometrically corrected light intensity matrix. After processing and noise filtering, the signal-to-noise ratio is improved to 40 dB, meeting the stability requirements of the radiometric correction input.

[0054] Location tag binding The radiation-corrected light intensity matrix and the structure location index are bound by location tags. A unique location tag is attached to each illumination data unit using a location tag encoding algorithm to form a location-bound illumination matrix.

[0055] Time series construction The location-bound illumination matrix is ​​processed to construct a temporal sequence. A temporal sampling and archiving algorithm is used to continuously sort the data according to the flight timestamp, generating a multi-location light field sampling temporal matrix. The accuracy of matching the separated single-band light intensity vector with the spatial location index is over 99%.

[0056] In step S2, the collaborative acquisition by a multispectral camera (visible light + near-infrared + short-wave infrared) and a high-sensitivity light sensor enables the system to simultaneously capture light field information across multiple wavelengths under low-light conditions, overcoming the performance bottleneck of a single visible light sensor in low-light environments. Radiometric correction eliminates the effects of sensor system noise and temperature drift, increasing the signal-to-noise ratio to 40dB, ensuring the accuracy and stability of the light field data, and providing high-precision time-series input for dynamic light field mapping.

[0057] Step S3: Generate dynamic light field mapping data Position-by-position difference processing Based on the multi-location light field sampling time series matrix, matrix difference processing is performed on the real-time illumination value corresponding to each structural location and the reference illumination value in the normalized illumination reference matrix. The brightness difference matrix is ​​calculated using a position-by-position brightness difference algorithm to obtain preliminary difference data reflecting the illumination change at a single location.

[0058] Location difference aggregation The brightness difference matrix and the location label index are subjected to location difference aggregation processing. The spatial label aggregation algorithm is used to calculate the statistical characteristics (mean, variance, extreme value) of the difference values ​​by grouping according to structural location, and a location difference feature matrix is ​​generated to highlight the spatial change pattern in low-light environment.

[0059] Time series difference trend calculation Based on the location difference feature matrix, the temporal difference trend calculation is performed. The temporal change trend analysis algorithm is used to extract the change slope (reflecting the rate of change of light) and fluctuation amplitude (reflecting the stability of light) of the difference value sequence at each location, and generate a temporal difference trend matrix to characterize the temporal dynamic characteristics of light change under low illumination environment.

[0060] Spatial topology weighted fusion The temporal difference trend matrix and the spatial layout label matrix are subjected to spatial weighting. The spatial topology weighted trend fusion algorithm is used to integrate the temporal change trend and spatial position relationship (relative height, span and proximity relationship) between adjacent structural locations to generate a spatial trend fusion difference matrix, which reflects the overall spatial dynamic pattern of light field changes under the bridge.

[0061] Dynamic light field interpolation Dynamic light field mapping is constructed based on the spatial trend fusion difference matrix. A dynamic light field interpolation algorithm is used to generate a continuous light field change surface between the difference values ​​at each position of the matrix, forming a dynamic light field mapping data matrix.

[0062] In step S3, the real-time acquired data and the baseline data are analyzed for differences at each location. Combined with temporal trend extraction and spatial topological weighting, the spatial and temporal characteristics of light field changes in low-illuminance environments are accurately captured. Dynamic light field interpolation generates a continuous light field change surface, enabling the system to accurately detect the trend of light changes under extremely low illuminance conditions, providing high-precision environmental change input conditions for the temporal obstacle prediction model.

[0063] Step S4: Collect point cloud data of obstacles at multiple locations LiDAR scanning Based on the structural location index matrix, sensor acquisition tasks are scheduled for the inspection path under the bridge. A high-resolution scanning mode is set using an airborne LiDAR, and a spatial coordinate mapping algorithm is employed to accurately locate the radar scan data to the corresponding structural positions, obtaining the LiDAR point cloud matrix for the structural positions.

[0064] Reflection intensity feature extraction Multi-beam reflection intensity acquisition and processing are performed on the lidar point cloud matrix of the structure location. The reflection intensity analysis algorithm is used to extract the surface echo feature parameter set (echo intensity, echo width, pulse shape) of the obstacle, and map and bind it to the structure location index to form a reflection intensity feature matrix.

[0065] Millimeter-wave radar scanning and multi-source fusion Based on the reflection intensity feature matrix, millimeter-wave radar scanning is performed on the same structural locations. A millimeter-wave penetration-echo analysis algorithm is used to obtain the obstacle point cloud penetration characteristic parameter matrix (penetration depth, backscattering coefficient), which is then fused with the reflection intensity feature matrix to generate a multi-source point cloud feature fusion matrix for the obstacle. Millimeter-wave radar retains its detection capability under low-light and dusty / foggy conditions, compensating for the limitations of visual attenuation.

[0066] Spatial filtering Spatial filtering is performed on the multi-source feature fusion matrix of obstacles. A three-dimensional spatial neighborhood filtering algorithm (radius search method, with the search radius set to 0.05m) is used to eliminate noise points and extract the boundary feature matrix of obstacles, thereby enhancing the spatial resolution accuracy of the dataset while preserving the integrity of the three-dimensional structure of the obstacles.

[0067] Location tag binding and grouping The obstacle boundary feature matrix and structural location index are bound together with location labels, and a location label encoding algorithm is used to attach a unique location label to each obstacle point cloud unit. A location index aggregation algorithm is then used to assemble obstacle data from different structural locations into a multi-location obstacle point cloud dataset, reducing the localization error to the centimeter level.

[0068] In step S4, the lidar provides high-precision three-dimensional spatial information, while the millimeter-wave radar retains its detection capability even under low-light and visibility-limited conditions such as dust and fog. The fusion of these two technologies enables the accurate extraction of the three-dimensional boundaries and penetration characteristics of obstacles. Three-dimensional spatial filtering eliminates noise points, and location tag binding and grouping ensure consistent registration between the obstacle point cloud and the spatial reference system, achieving a positioning error at the centimeter level. This provides a high-precision spatial obstacle distribution input for the temporal obstacle prediction model.

[0069] Step S5: Predicting Temporal Barriers Timestamp synchronization alignment The dynamic light field mapping data matrix and the multi-location obstacle point cloud dataset are time-stamped and synchronized. A time-series data alignment algorithm is used to align the two types of input data in terms of position and time according to a unified flight time axis, generating a time-series aligned multi-source input matrix.

[0070] Feature domain mapping The temporally aligned multi-source input matrix is ​​processed by feature domain mapping. The spatial brightness difference features in the dynamic light field mapping and the boundary structure features in the obstacle point cloud are converted into a unified feature vector space using a multi-source unified feature matrix, thus eliminating the influence of inconsistent feature dimensions.

[0071] Temporal convolution fusion Based on the multi-source unified feature matrix, temporal feature fusion processing is performed. The temporal convolution fusion algorithm (with the convolution kernel size set to 3×3 and the stride to 1) is used to perform position-by-position fusion calculation of the light field change trend features and obstacle position dynamic features according to the structural position labels, generating a position dynamic fusion feature matrix that includes both illumination change factors and obstacle dynamic factors.

[0072] Spatial topology weighting Spatial topological weighting is performed on the location dynamic fusion feature matrix. The topological weighting trend optimization algorithm is used to adjust the joint feature values ​​spatially by combining the relative height, span and proximity relationship between structural locations, generating a spatial relationship weighted fusion feature matrix, which enhances the prediction model's ability to perceive spatial dependence.

[0073] Model preprocessing The spatial relationship weighted fusion feature matrix is ​​input into the temporal obstacle prediction model. The joint features are standardized and formed into a fixed-length temporal input segment by using feature normalization and sliding window sequence generation algorithm (window length is set to 10 time steps and step size is 1) to obtain a standardized temporal fusion feature sequence.

[0074] In step S6, the timestamps of the dynamic light field mapping data and the obstacle point cloud data are synchronized and the feature domains are mapped, realizing a unified representation of multi-source heterogeneous data. Temporal convolutional fusion and spatial topological weighting enable the prediction model to simultaneously perceive changes in the ambient light field and the dynamic behavior of obstacles, achieving millisecond-level updates. The prediction calculation can issue an early warning seconds before the obstacle appears, effectively shortening the reaction time.

[0075] Step S6: Generate flight path attitude adjustment parameters and perform fine-tuning. Probability Extraction Based on the prediction results of the spatial relationship weighted fusion feature matrix, obstacle occurrence probability extraction processing is performed. The probability extraction algorithm is used to read the obstacle occurrence probability value from the prediction sequence position by position to form an obstacle probability value matrix.

[0076] Threshold determination The obstacle probability value matrix is ​​compared with a preset safety threshold (set to 0.75 in this embodiment) to perform probability determination processing. The probability threshold comparison algorithm is used to calculate the determination mark matrix position by position, and a trigger mark is generated at the position where the probability value exceeds the threshold.

[0077] Trend correlation Based on the judgment mark matrix, position change trend correlation processing is performed. The trend correlation analysis algorithm is used to register the trigger mark with the dynamic position offset in the position change trend matrix to generate the trigger position dynamic offset matrix.

[0078] Attitude adjustment calculation The dynamic offset matrix of the trigger position and the flight attitude reference parameter matrix are used to calculate the attitude adjustment. An attitude parameter calculation algorithm is employed to calculate the roll, pitch, and velocity adjustments position by position. In this embodiment, the roll adjustment ΔRoll is +3.5°, and the pitch adjustment ΔPitch is... 2.0°, speed adjustment ΔV is 0.8 m / s, generate the position and attitude adjustment parameter matrix.

[0079] Parameter grouping summary Position grouping is performed based on the position and attitude adjustment parameter matrix. The adjustment parameters are summarized by structural position grouping using the position index aggregation algorithm to generate a grouped attitude adjustment parameter table.

[0080] Flight attitude fine-tuning control (1) Call the attitude adjustment interface of the flight control system to perform control task initialization. The interface ID is set to 0x12, and the control permission level is set to 4 (highest permission). Map the roll, pitch, and speed adjustment parameters to the flight control hardware driver interface to generate the interface mapping parameter matrix [[3.5, 2.0, 0.8]. In the flight attitude initialization algorithm, the attitude reference value is set to the current stable flight value, and the execution priority weights are Roll 0.5, Pitch 0.3, and V 0.2. The task queue construction algorithm places Roll adjustment at the head of the queue according to the execution order, followed by Pitch adjustment, and finally speed fine-tuning, forming a task queue matrix: [[Roll,3.5],[Pitch, 2.0],[V, 0.8]).

[0081] (2) The adjustment amount is written into the flight control system cache in sequence using the real-time loading algorithm of flight attitude parameters to form an attitude adjustment cache matrix.

[0082] (3) The roll, pitch and speed adjustment quantities are converted into control signal sequences that conform to the flight controller communication protocol by using the flight attitude control signal synthesis algorithm to generate the attitude control signal matrix.

[0083] (4) Input the attitude control signal matrix into the flight control execution unit, use the flight actuator drive algorithm to perform synchronous fine-tuning processing on each attitude axis of the UAV, adjust the output frequency of the control signal in real time according to the spatial position parameters of the obstacle, and generate a fine-tuning execution feedback data matrix.

[0084] (5) The flight attitude execution result is matched with the bridge structure spatial layout label using a spatial safety distance calculation algorithm to generate a safety distance judgment matrix. If the safety threshold judgment is qualified, the current flight path is maintained; if it is not qualified, a secondary attitude fine-tuning task is triggered.

[0085] (6) Use the attitude stabilization adjustment algorithm to make minor corrections to the roll, pitch and velocity parameters, and generate the attitude stabilization parameter matrix as the attitude adjustment reference input for the next moment.

[0086] In step S6, the results show that the roll attitude deviation was 4° before fine-tuning and was reduced to within 0.5° after fine-tuning; the pitch attitude deviation was reduced from... Adjust to 2.5° The speed deviation was reduced to within 0.05 m / s, and the obstacle avoidance success rate reached over 97%. The complete closed-loop implementation from prediction results to flight control commands ensured the speed and accuracy of active obstacle avoidance, while the safe distance determination and attitude stabilization ensured that the safe distance from the bridge structure was consistently met during the obstacle avoidance process.

[0087] Step S7: Route Restoration and Smooth Connection Probability dynamic monitoring Based on the standardized temporal fusion feature sequence of the temporal obstacle prediction model, the probability matrix of obstacle occurrence is extracted time slice by time using the probabilistic time series analysis algorithm.

[0088] Trend correlation fusion The obstacle occurrence probability matrix and the dynamic light field mapping data matrix are subjected to trend correlation processing. The light field-probability joint trend analysis algorithm is used to fuse the probability time series features and the brightness change trend matrix into a light field probability joint trend matrix, which simultaneously depicts the dual-factor coupling state of improved ambient lighting and reduced obstacle risk.

[0089] Dual-condition threshold determination The state determination process is performed based on the joint trend matrix of light field probability. A composite threshold comparison algorithm is used to make dual-condition determinations on the obstacle occurrence probability (threshold 0.35) and the illumination improvement coefficient (threshold 0.6). When both satisfy the preset safety threshold, a route switching trigger marker matrix is ​​generated.

[0090] Location index registration The position index registration process is performed on the original planned route parameter group corresponding to the route switching trigger mark matrix and the normalized illumination reference matrix. The trigger mark position is mapped to the original planned route parameter matrix using the route parameter registration algorithm to form the route recovery position parameter matrix.

[0091] Route switching control Based on the flight path recovery position parameter matrix, the flight path switching control signal generation and processing is performed. The flight path switching control signal synthesis algorithm is used to convert the original planned flight path parameter group into a switching command matrix that conforms to the flight controller communication protocol, which is then loaded into the flight control cache to drive the UAV to smoothly switch to the global flight path state.

[0092] Switching effect feedback monitoring A flight status feedback analysis algorithm is used to analyze the differences between the switched flight attitude data and the original planned route parameter set, generating a route switching stability judgment matrix. When the judgment result is lower than the stability threshold, a secondary attitude fine-tuning task is triggered to ensure that the global route control remains smooth and continuous during the switching process.

[0093] In step S7, a dual-condition threshold judgment mechanism (reduced obstacle probability and improved light field conditions) ensures the accuracy and safety of route recovery, avoiding erroneous switching. The route switching stability judgment pass rate reaches over 95%, ensuring a smooth switching between local obstacle avoidance actions and the original planned route. Real-time feedback monitoring and secondary fine-tuning mechanisms further guarantee the smoothness of the switching process, improving the continuity of the inspection path and data acquisition efficiency.

[0094] Step S8: Model Parameter Update and Self-Learning Data Synchronous Collection Data synchronization acquisition and processing are performed based on the flight path attitude adjustment parameter group and dynamic light field mapping data matrix. The flight event synchronization marking algorithm is used to associate attitude adjustment events with corresponding environmental change data with timestamps to generate an attitude-environment synchronization record matrix.

[0095] Feature extraction Feature extraction processing is performed on the attitude-environment synchronization recording matrix. The roll, pitch, velocity adjustment, light field brightness difference features, and obstacle spatial position features are uniformly converted into model training feature vector matrix using a multi-source feature parsing algorithm.

[0096] Feature normalization The maximum-minimum normalization algorithm is used to scale all feature values ​​to a uniform standardized range of [0,1] to generate a standardized feature matrix, thereby eliminating the impact of feature scale differences on model parameter adjustment.

[0097] Time series sample construction The time series sample construction process is performed based on the standardized feature matrix. The sliding window sample generation algorithm (with a window length set to 20 time steps) is used to divide the continuously recorded data into a fixed-length time series sample sequence to generate a time series training sample matrix.

[0098] Incremental learning and updating The time-series training sample matrix is ​​input into the time-series obstacle prediction model to perform parameter update processing. An incremental learning training algorithm (learning rate set to 0.001, batch size set to 32) is used to iteratively optimize the model weights and time-series feature mapping coefficients to generate the updated time-series obstacle prediction model parameter set.

[0099] Model performance evaluation The updated time-series obstacle prediction model parameter set is subjected to model performance evaluation processing. The accuracy score of future path obstacle prediction and the response coefficient of environmental change are calculated using a dual-index evaluation algorithm of prediction accuracy and environmental adaptability. A performance evaluation result matrix is ​​generated and archived in pairs with the attitude-environment synchronization record matrix.

[0100] In step S8, a closed-loop record-process-update mechanism enables the prediction model to continuously adapt to dynamic environmental changes and optimize obstacle avoidance decisions. Through incremental learning training algorithms, the model weights and time-series feature mapping coefficients are continuously iteratively optimized as tasks accumulate, continuously improving prediction accuracy and environmental adaptability, and reducing prediction errors by more than 10% in the long term. Performance evaluation and archiving form a closed-loop performance record of the continuous optimization process, reducing manual intervention and improving the reliability of unmanned inspection.

[0101] Example 2 This embodiment provides a bridge inspection drone route planning system, which is used to implement the bridge inspection drone route planning method as described in Embodiment 1.

[0102] The system includes an unmanned aerial vehicle (UAV) platform, an airborne sensing unit, an airborne data processing unit, a flight control unit, a ground management unit, and a communication unit.

[0103] The unmanned aerial vehicle (UAV) platform is a multi-rotor UAV, with a gimbal mounting base under its fuselage for securing the onboard sensing unit. The onboard sensing unit includes a multispectral camera, a high-sensitivity light sensor, a lidar, and a millimeter-wave radar. The multispectral camera and high-sensitivity light sensor are mounted adjacent to each other under the nose of the UAV, the lidar is mounted in the center of the UAV's underside, and the millimeter-wave radars are mounted forward and to the sides of the UAV.

[0104] The airborne data processing unit is an embedded AI computing platform, which integrates an FPGA acceleration chip and a GPU computing core. The airborne data processing unit is connected to a multispectral camera, a high-sensitivity light sensor, a lidar, a millimeter-wave radar, and a flight control unit.

[0105] The flight control unit includes a flight controller, an inertial measurement unit (IMU), an electronic speed controller, and actuators. The flight controller is communicatively connected to the airborne data processing unit, the IMU, and the electronic speed controller, and the electronic speed controller is electrically connected to each rotor motor.

[0106] The ground management unit is a ground station computer, which is wirelessly connected to the airborne data processing unit through a communication unit and is used to issue initial route planning parameters and receive inspection data.

[0107] The multispectral camera is equipped with three independent imaging channels: visible light band (400~700nm), near-infrared band (700~1100nm), and short-wave infrared band (1100~1700nm). Each channel acquires data synchronously at a frame rate of no less than 30fps.

[0108] The high-sensitivity light sensor is a silicon-based photodiode array sensor with a light-sensing range covering 300~1700nm, a minimum illuminance detection limit of 0.01lx, and a sampling frequency of not less than 100Hz.

[0109] The lidar is a 16-line rotating lidar with a ranging accuracy of ±2cm, an effective ranging range of 0.5~100m, and a scanning frequency of 20Hz, used to acquire high-precision three-dimensional point cloud data of the structure under the bridge.

[0110] The millimeter-wave radar is a 77GHz frequency modulated continuous wave (FMCW) radar, which has the ability to penetrate low-light and dusty / foggy environments, with a ranging accuracy of ±5cm and an effective detection range of 0.2~50m, and is used to obtain the penetration characteristic parameters of obstacles.

[0111] The airborne data processing unit has a built-in memory that stores three-dimensional point cloud data of different structural locations of the bridge, low-light illumination distribution data, material type and spatial layout labels, as well as executable program modules such as illumination reference model initialization algorithm, spectral feature separation algorithm, temporal convolution fusion algorithm, and incremental learning training algorithm.

[0112] After the system starts, the airborne data processing unit first loads the bridge 3D point cloud data and low-light illumination distribution data pre-stored in the memory, performs structural position index matrix generation, material spectral characteristic parameter set extraction and spatial layout label generation, establishes illumination reference model and performs normalization processing to generate normalized illumination reference matrix.

[0113] After the drone takes off and enters the inspection area under the bridge, the multispectral camera and high-sensitivity light sensor continuously collect real-time multi-band light signals at each structural location according to the preset sampling frequency. After receiving the light signals, the airborne data processing unit sequentially performs spectral decomposition, radiometric correction, location tag binding, and time sequence construction to generate a multi-location light field sampling time sequence matrix.

[0114] Meanwhile, the lidar performs a high-resolution scan of the structure under the bridge according to the set scanning mode, while the millimeter-wave radar simultaneously performs a penetration scan of the same structural location. After receiving the lidar point cloud data and the millimeter-wave radar echo data, the airborne data processing unit performs reflection intensity feature extraction, multi-source point cloud feature fusion, three-dimensional spatial filtering, and location label binding to generate a multi-location obstacle point cloud dataset.

[0115] The airborne data processing unit synchronizes and aligns the dynamic light field mapping data with the multi-location obstacle point cloud dataset along a unified flight time axis, and then inputs the data into a temporal obstacle prediction model built into the airborne data processing unit. This model employs an architecture combining a Long Short-Term Memory (LSTM) network and a Graph Attention Network (GAT). The LSTM layer captures the temporal dependency between light field changes and obstacle motion, while the GAT layer models the spatial topological dependencies between different structural locations. After forward computation, the model outputs predictions of obstacle occurrence probabilities and positional change trends at each structural location along the future flight path.

[0116] The airborne data processing unit performs threshold determination on the prediction results: when the probability of an obstacle appearing at any structural position is higher than a preset threshold (0.75 in this embodiment), the airborne data processing unit automatically calculates the roll, pitch and speed adjustment amounts, generates a flight attitude adjustment parameter set, and sends the parameter set to the flight control unit.

[0117] After receiving the flight path attitude adjustment parameter set, the flight control unit writes the roll, pitch, and speed adjustments into the flight controller's buffer. The flight controller then converts these adjustments into PWM control signals and outputs them to the electronic speed controller (ESC). The ESC drives each rotor motor to perform synchronous fine-tuning. Simultaneously, based on real-time obstacle distance data fed back by millimeter-wave radar, the flight controller dynamically adjusts the control signal output frequency to ensure a preset safe distance from the bridge structure (0.5m in this embodiment).

[0118] During attitude fine-tuning, the flight control unit uses the IMU to collect real-time data on the UAV's three-axis angular velocity and linear acceleration. Based on this data, the flight controller performs attitude stabilization closed-loop control, making minor corrections to roll, pitch, and velocity parameters.

[0119] During flight, the onboard data processing unit continuously monitors the output of the temporal obstacle prediction model. When the probability of an obstacle appearing drops below a preset recovery threshold (0.35 in this embodiment) and the illumination improvement coefficient in the dynamic light field mapping data is higher than a preset threshold (0.6 in this embodiment), the system automatically generates a route switching trigger marker, maps the marker position to the original planned route parameter matrix, generates a route recovery position parameter group, and sends it to the flight control unit to drive the UAV to smoothly switch to the original planned route.

[0120] After each flight attitude adjustment event, the onboard data processing unit automatically performs data recording and model updates: it timestamps the attitude adjustment event with the corresponding environmental change data, extracts roll, pitch, and speed adjustments, light field differences, and obstacle position features, converts these into model training feature vectors, and uses an incremental learning algorithm to iteratively optimize the weights and feature mapping coefficients of the temporal obstacle prediction model. The updated model parameters are stored in the onboard memory for prediction calculations on the next flight.

[0121] Furthermore, it should be noted that the use of terms such as "first" and "second" to define components is merely for the purpose of distinguishing the corresponding components. Unless otherwise stated, these terms have no special meaning and therefore should not be construed as limiting the scope of protection of this application. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for planning the flight path of a bridge inspection drone, characterized in that, include: S1. Before the drone takes off, based on the three-dimensional point cloud data of different structural locations of the bridge, low-light illumination distribution data, material type and spatial layout labels, establish an illumination reference model, and normalize the illumination reference model to generate a normalized illumination reference matrix. S2. During flight, real-time ambient light field data at different locations under the bridge are collected using a multispectral camera and a light sensor to form a multi-location light field sampling sequence. S3. Perform difference calculation between the multi-location light field sampling sequence and the normalized illumination reference matrix to generate dynamic light field mapping data; S4. Use lidar and millimeter-wave radar to collect real-time obstacle point cloud data at different structural locations under the bridge to form a multi-location obstacle point cloud dataset. S5. Simultaneously input the dynamic light field mapping data and the multi-location obstacle point cloud dataset into the temporal obstacle prediction model to generate prediction results of the probability of obstacle appearance and the trend of position change on the future flight path. S6. When the probability of an obstacle appearing in the prediction result is higher than the preset threshold, generate a flight attitude adjustment parameter group that includes roll, pitch and speed adjustment, and execute flight attitude fine-tuning control. S7. When the probability of obstacles appears decreases and the light field conditions improve, switch to the original planned route parameter group corresponding to the normalized illumination reference matrix.

2. The bridge inspection drone route planning method according to claim 1, characterized in that: Following step S7, the procedure also includes: S8. Record each flight path attitude adjustment and corresponding environmental data, and update the parameters of the timing obstacle prediction model.

3. The method for planning the flight path of a bridge inspection drone according to claim 1, characterized in that: Step S1 includes: generating a structure location index matrix based on 3D point cloud data; Material type acquisition is performed on the structural location index matrix to obtain the material spectral characteristic parameter set and bind it to the corresponding structural location index; The low-illuminance light distribution data is registered with the structural location index matrix and the material spectral characteristic parameter set to form a three-dimensional mapping table of location-material-illuminance. Spatial layout labels are generated for the 3D mapping table, and spatial topology analysis is used to calculate the relative height, span, and proximity of each structure location. The input condition matrix for the illumination reference model is generated based on the 3D mapping table and spatial layout labels. The material type acquisition includes: using a spectral feature separation algorithm to perform band decomposition and feature extraction on the original spectral signal to obtain the material feature vector; and using a material spectral characteristic parameter calculation algorithm to map the material feature vector into a set of material spectral characteristic parameters including reflectance coefficient, absorption coefficient, peak position, and peak width.

4. The bridge inspection drone route planning method according to claim 3, characterized in that: In step S1, the normalization process includes: Calculate the normalized coefficient matrix of material reflectance based on the reflectance coefficient in the set of material spectral characteristic parameters; The normalized coefficient matrix is ​​multiplied point-by-point with the input condition matrix of the illumination reference model to obtain the reflectivity-corrected illumination matrix. The position illumination mean is normalized by normalizing the reflectivity-corrected illumination matrix to obtain the position brightness normalized matrix; The position brightness normalization matrix is ​​combined with the spatial layout labels to perform spatial topology weighted average smoothing to obtain a smoothed illumination matrix. The smoothed illumination matrix is ​​normalized by global maximum and minimum values ​​to generate a normalized illumination reference matrix.

5. The bridge inspection drone route planning method according to claim 1, characterized in that: Step 2 includes: The light field acquisition task is planned based on the structure location index in the normalized illumination reference matrix, and the real-time multi-band illumination signal matrix of each structure location is obtained using a multispectral camera. The multi-band illumination signal matrix is ​​spectrally decomposed into single-band light intensity vectors; radiometric correction is performed on the single-band light intensity vectors to eliminate the effects of sensor system noise and temperature drift. The radiation-corrected light intensity data is bound to the structural location index with location tags to form a location-bound illumination matrix; The location-bound illumination matrix is ​​continuously sorted according to the flight timestamp to generate a multi-location light field sampling time sequence matrix.

6. The method for planning the flight path of a bridge inspection drone according to claim 1, characterized in that: Step S3 includes: A position-by-position difference processing is performed on the multi-position light field sampling time sequence matrix and the normalized illumination reference matrix to obtain the brightness difference matrix; Calculate the statistical characteristics of the differences by grouping according to structural location, and generate a location difference feature matrix; Extract the temporal change slope and fluctuation amplitude from the location difference feature matrix to generate a temporal difference trend matrix; By combining spatial layout labels, a spatial topological weighted fusion of the temporal difference trend matrix is ​​performed to generate a spatial trend fusion difference matrix. Dynamic light field interpolation is performed on the spatial trend fusion difference matrix to form a dynamic light field mapping data matrix.

7. The bridge inspection drone route planning method according to claim 4, characterized in that: Step S4 includes: The location of the structure under the bridge was scanned using lidar to obtain the lidar point cloud matrix of the structure location; Extract the reflection intensity features of the lidar point cloud matrix to form a reflection intensity feature matrix; Millimeter-wave radar is used to scan the same structural location to obtain the penetration characteristic parameter matrix, and multi-source point cloud feature fusion is performed with the reflection intensity feature matrix to generate the obstacle multi-source feature fusion matrix. Three-dimensional spatial filtering is performed on the multi-source feature fusion matrix to extract the obstacle boundary feature matrix; The boundary feature matrix and the structural location index are bound to location labels and grouped to form a multi-location obstacle point cloud dataset.

8. The bridge inspection drone route planning method according to claim 1, characterized in that: Step S5 includes: The dynamic light field mapping data and the multi-location obstacle point cloud dataset are time-stamped and aligned to generate a time-aligned multi-source input matrix. The light field difference features and obstacle boundary structure features in the time-aligned multi-source input matrix are converted into a unified feature vector space to form a multi-source unified feature matrix. Temporal convolutional fusion is performed on the multi-source unified feature matrix to generate a location-dynamically fused feature matrix; By combining the relative height, span, and proximity relationships between structural locations, the location dynamic fusion feature matrix is ​​spatially topologically weighted to generate a spatial relationship weighted fusion feature matrix; The spatial relationship weighted fusion feature matrix is ​​standardized and a fixed-length temporal input segment is generated, which is then input into the temporal obstacle prediction model.

9. The method for planning the flight path of a bridge inspection drone according to claim 1, characterized in that: Step S6 includes: The probability values ​​of obstacle occurrence are extracted from the prediction results at each position to form an obstacle probability value matrix; The probability value matrix is ​​compared with a preset safety threshold to generate a judgment mark matrix; The judgment mark matrix is ​​registered with the position change trend matrix to generate the trigger position dynamic offset matrix; Based on the dynamic offset matrix of the trigger position and the reference parameter matrix of the flight attitude, the roll adjustment, pitch adjustment and velocity adjustment are calculated position by position to generate a position and attitude adjustment parameter matrix; the adjustment parameters are then grouped and summarized according to structural position to generate a grouped attitude adjustment parameter table. Step S7 includes: Extract the obstacle occurrence probability matrix based on the prediction results for each time slice; By trend correlation and fusion of the probability matrix and dynamic light field mapping data, a joint trend matrix of light field probability is generated. The joint trend matrix is ​​subjected to a dual-condition threshold determination of obstacle occurrence probability and illumination improvement coefficient to generate a route switching trigger marker matrix; The trigger marker matrix is ​​mapped to the original planned route parameter matrix to generate the route recovery position parameter matrix; The original planned route parameter set is converted into a switching command and loaded into the flight control cache to execute the route switch; The difference between the switched flight attitude and the original planned route parameters is analyzed. If the stability is lower than the threshold, a secondary attitude fine-tuning is triggered.

10. A bridge inspection drone flight path planning system, characterized in that: This method is used to implement the bridge inspection drone route planning method as described in any one of claims 1-9.

Images