Automatic identification and inspection method of bridge defects based on deep learning

Abstract

The present invention relates to the field of bridge inspection technology, and specifically to a deep learning-based automatic bridge defect identification and inspection method, comprising the following steps: S1, multimodal data acquisition: acquiring multimodal data of the bridge surface and internal structure; S2, multimodal data preprocessing: generating a spatiotemporally aligned enhanced dataset; S3, feature fusion and extraction: generating a fused feature map; S4, adaptive optimization of the feature map: adjusting the inter-layer connection weights of the network based on real-time environmental parameters; S5, defect location and type identification: locating the spatial coordinates of the defect and synchronously outputting the probability distribution of the defect type; S6, defect analysis and report generation: generating an inspection report including defect evolution trend analysis based on the results of defect location and type identification. This invention improves bridge inspection efficiency, reduces labor costs, and further promotes the intelligent upgrade of bridge management.

Description

Technical Field

[0001] The present invention relates to the field of bridge detection technology, and in particular to a method for automatic identification and inspection of bridge defects based on deep learning. Background Art

[0002] With the rapid development of the transportation industry, bridges, as a key component of transportation infrastructure, bear huge traffic flows and ensure people's travel safety. The health of bridges directly affects traffic safety and bridge service life. Therefore, regular bridge inspections and maintenance are particularly important. Traditional bridge inspection methods mainly rely on manual inspections. Workers conduct close observation and inspections by climbing and scaling. This method is not only labor-intensive and inefficient, but also has certain safety hazards and high personnel costs. It is difficult to fully cover all parts of the bridge structure, especially some difficult-to-access parts.

[0003] At present, with the continuous development of drone technology and sensor equipment, drone-based bridge inspection technology has gradually been applied, but the existing drone bridge inspection technology still has some shortcomings. First, in terms of data collection, although drones can be equipped with a variety of sensors, the data collected by different sensors often have the problem of temporal and spatial asynchrony, which makes data fusion and subsequent analysis very difficult. Secondly, most of the existing defect identification methods rely on manual or simple rule algorithms, lack intelligence and automation, and cannot fully utilize the advantages of technologies such as deep learning, resulting in the accuracy and efficiency of defect identification unable to meet actual needs. Finally, existing technologies generally lack defect evolution trend analysis and intelligent decision support, making it difficult to predict the development trend of defects in advance and unable to provide effective decision support for bridge maintenance and repair.

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and propose a method for automatic identification and inspection of bridge defects based on deep learning, so as to improve the automation and intelligence level of bridge inspection, reduce labor costs, extend the service life of bridges, and ensure traffic safety. Summary of the Invention

[0005] The present invention provides a method for automatic identification and inspection of bridge defects based on deep learning.

[0006] The deep learning-based automatic bridge defect recognition and inspection method includes the following steps:

[0007] S1, multimodal data acquisition: The multi-dimensional image acquisition equipment carried by the UAV is used to obtain multimodal data of the bridge surface and internal structure, including visible light images, infrared thermal imaging data, and 3D laser point cloud data;

[0008] S2, multimodal data preprocessing: a spatiotemporal synchronization enhancement algorithm is used to preprocess the multimodal data to generate a spatiotemporally aligned enhanced dataset;

[0009] S3, feature fusion and extraction: The spatiotemporally aligned augmented dataset is fed into a multimodal feature fusion network, which extracts defect features of different dimensions through a cross-scale convolution kernel group to generate a fused feature map.

[0010] S4, adaptive optimization of feature maps: Adaptively optimize the fused feature maps based on a dynamic weight allocation mechanism, and adjust the connection weights between network layers according to real-time environment parameters;

[0011] S5, defect localization and type identification: The optimized fusion feature map is input into the bidirectional attention localization mechanism to locate the spatial coordinates of the defect and simultaneously output the probability distribution of the defect type;

[0012] S6, Defect Analysis and Report Generation: Based on the results of defect location and type identification, an inspection report including defect evolution trend analysis is generated.

[0013] Optionally, the multimodal data acquisition in S1 includes:

[0014] S11, UAV equipment selection and configuration: Select a UAV platform and equip it with multi-dimensional image acquisition equipment, including a visible light image acquisition unit, an infrared thermal imaging unit, and a 3D laser point cloud scanner;

[0015] S12, Bridge Surface and Internal Structure Data Collection: By controlling the drone to fly to different positions and angles of the bridge, multimodal data of the bridge surface and internal structure are collected, and a data set including visible light images, infrared thermal imaging data, and 3D laser point cloud data is obtained.

[0016] Optionally, the multimodal data preprocessing in S2 includes:

[0017] S21, Spatiotemporal Data Synchronization Analysis: Analyze the temporal and spatial dimensions of multimodal data, and identify synchronization deviations between different data sources by calculating the timestamps and spatial coordinates of each modality;

[0018] S22, Application of spatiotemporal enhancement algorithm: Enhance multimodal data through spatiotemporal synchronization enhancement algorithm to improve the spatiotemporal alignment accuracy of each modality data and generate enhanced data set D aug ;

[0019] S23, generate a spatiotemporal aligned dataset: based on the generated enhanced dataset D aug , generate a spatiotemporal aligned augmented dataset D aligned ;

[0020] S24, Data quality assessment: Evaluate the spatiotemporal alignment accuracy of the dataset by calculating the mean alignment error (MAE).

[0021] Optionally, the feature fusion and extraction in S3 includes:

[0022] S31, inputting the spatiotemporally aligned augmented dataset: inputting the spatiotemporally aligned augmented dataset into the multimodal feature fusion network;

[0023] S32, feature extraction using cross-scale convolution kernel groups: Through the cross-scale convolution kernel group in the multimodal feature fusion network, defect feature maps F of different scales and dimensions are extracted. i ;

[0024] S33, fusion of different modal features: the feature maps F1, F2, ..., F from different modalities are combined n Perform fusion to generate a multimodal fusion feature map F fusion .

[0025] Optionally, the adaptive optimization feature map in S4 includes:

[0026] S41, real-time environmental parameter collection: collect current environmental parameters E in real time through environmental sensors, including temperature, humidity, and light intensity;

[0027] S42, dynamic weight allocation mechanism design: according to the real-time environment parameter E, the weight of each layer of the network is adjusted through an adaptive mechanism;

[0028] S43, optimize the weight connection of the fusion feature map: dynamically optimize the connection weights of each network layer according to the real-time environment parameters, and output the adaptively optimized fusion feature map F opt .

[0029] Optionally, the defect location and type identification in S5 includes:

[0030] S51, defect spatial coordinate positioning: the adaptively optimized fusion feature map is input into the bidirectional attention positioning mechanism to locate the spatial coordinates of the defect;

[0031] S52, defect type identification and probability distribution output: Based on the spatial coordinates of the located defects, the bidirectional attention mechanism identifies the type of each defect and outputs the probability distribution of the defect type.

[0032] Optionally, the defect spatial coordinate positioning in S51 includes:

[0033] S511, Bidirectional Attention Mechanism Calculation: In the bidirectional attention mechanism, the adaptively optimized fusion feature map F is forward propagated. optProcess and calculate the importance weight A of each position i,j , and adjust and strengthen the weights of the defective areas through back propagation;

[0034] S512, defect space coordinate calculation: according to the calculated attention weight A i,j , select the area with the highest weight as the spatial coordinate C of the defect.

[0035] Optionally, the defect type identification and probability distribution output in S52 includes:

[0036] S521, extract the defect area based on the spatial coordinates: based on the acquired spatial coordinates of the defect C = (x c ,y c ), from the adaptively optimized fusion feature map F opt Extract the feature map F corresponding to the defect area region ;

[0037] S522, defect type identification: extract the defect area feature map F region Input into the bidirectional attention mechanism, perform type recognition based on the characteristics of the defect area, and calculate the probability P of each defect type through forward propagation t ;

[0038] S523, output defect type probability distribution: according to the calculated defect type probabilities P1, P2, ..., P T , output the type of each defect area and its corresponding probability distribution.

[0039] Optionally, the defect analysis and report generation in S6 includes:

[0040] S61, Defect Evolution Trend Analysis: Based on historical multimodal data and the spatial coordinates and type identification results of the current defect, the defect evolution trend is analyzed and the future development trend of the defect is predicted using a regression analysis model;

[0041] S62, generating an inspection report: combining the results of defect location and type identification and defect evolution trend analysis, generating an inspection report including the defect evolution trend.

[0042] Optionally, generating the inspection report in S62 includes:

[0043] S621, Defect Location and Type: Lists the location coordinates of each defect and the probability distribution of type identification;

[0044] S622, Defect Evolution Trend: Shows the evolution trend of defects at different time points and predicts the future state changes of defects;

[0045] S623, Maintenance Recommendations: Based on the type and evolution trend of the defects, make corresponding maintenance or reinforcement recommendations, including crack repair, corrosion prevention and reinforcement, local reinforcement and replacement, bridge deck or bearing replacement, extended monitoring and tracking, and structural reconstruction and renovation.

[0046] Beneficial effects of the present invention:

[0047] The present invention, through the comprehensive use of drones, multimodal data acquisition technology and spatiotemporal synchronization enhancement algorithms, can efficiently and accurately collect multimodal data of the bridge surface and internal structure, and perform spatiotemporal alignment and enhancement processing, providing high-quality input data for subsequent defect identification and analysis. Through the fusion and precise alignment of multiple modal data, the integrity and complementarity of the data are significantly improved, thereby effectively enhancing the deep learning model's ability to identify bridge defects, ensuring stable and efficient automated inspections in various environments.

[0048] This invention introduces an adaptive optimization mechanism, adjusts the weights of the feature map based on real-time environmental parameters, improves the adaptability and accuracy of the model in complex environments, and combines the two-way attention mechanism to accurately locate the spatial coordinates of defects, identify the type of defects, and output the probability distribution. Through this adaptive optimization and precise positioning, efficient detection and classification of bridge defects are achieved, which not only improves the recognition accuracy, but also provides a reliable basis for subsequent maintenance decisions, further promoting the development of intelligent and automated bridge inspections.

[0049] The present invention can automatically generate inspection reports containing defect evolution trends through defect type identification and evolution trend analysis, providing bridge managers with comprehensive and accurate defect assessments and maintenance recommendations. By combining the type and evolution trend of defects, the report can provide early warning of potential structural risks, helping managers make timely repair or reinforcement decisions, optimize resource allocation, extend the service life of bridges, and ensure the safety and reliability of bridges. This intelligent inspection and report generation mode has greatly improved bridge inspection efficiency, reduced labor costs, and further promoted the intelligent upgrade of the bridge management field. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 Schematic diagram of the identification and inspection method according to an embodiment of the present invention;

[0052] Figure 2Schematic diagram of the adaptive optimization feature map according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. It is also noted that, to provide a more detailed description, the following embodiments are best and preferred embodiments, and those skilled in the art may employ alternative methods for implementing certain known technologies. Furthermore, the accompanying drawings are intended only to provide a more detailed description of the embodiments and are not intended to limit the present invention.

[0054] It should be noted that references in the specification to "one embodiment," "an embodiment," "an exemplary embodiment," "some embodiments," etc. indicate that the described embodiments may include specific features, structures, or characteristics, but not every embodiment necessarily includes such specific features, structures, or characteristics. In addition, when specific features, structures, or characteristics are described in conjunction with an embodiment, it is within the knowledge of persons skilled in the relevant art to implement such features, structures, or characteristics in conjunction with other embodiments (whether or not explicitly described).

[0055] In general, terms can be understood, at least in part, from their use in context. For example, depending at least in part on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending at least in part on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0056] like Figure 1-Figure 2 As shown in FIG, the automatic identification and inspection method of bridge defects based on deep learning includes the following steps:

[0057] S1, multimodal data acquisition: The multi-dimensional image acquisition equipment carried by the UAV is used to obtain multimodal data of the bridge surface and internal structure, including visible light images, infrared thermal imaging data, and 3D laser point cloud data;

[0058] S2, multimodal data preprocessing: a spatiotemporal synchronization enhancement algorithm is used to preprocess the multimodal data to generate a spatiotemporally aligned enhanced dataset;

[0059] S3, feature fusion and extraction: The spatiotemporally aligned augmented dataset is fed into a multimodal feature fusion network, which extracts defect features of different dimensions through a cross-scale convolution kernel group to generate a fused feature map.

[0060] S4, adaptive optimization of feature maps: Adaptively optimize the fused feature maps based on a dynamic weight allocation mechanism, and adjust the connection weights between network layers according to real-time environment parameters to improve model recognition accuracy;

[0061] S5, defect localization and type identification: The optimized fusion feature map is input into the bidirectional attention localization mechanism to locate the spatial coordinates of the defect and simultaneously output the probability distribution of the defect type;

[0062] S6, Defect Analysis and Report Generation: Based on the results of defect location and type identification, an inspection report including defect evolution trend analysis is generated;

[0063] Through the above content, efficient and accurate bridge defect identification and positioning are achieved. It can fully utilize the multi-dimensional image data collected by drones to automatically analyze and evaluate the health status of bridge structures, provide real-time and accurate defect detection results, and generate inspection reports with evolution trend analysis. This improves the automation and intelligence level of bridge inspections, reduces labor costs, and improves the efficiency and safety of bridge management and maintenance.

[0064] Multimodal data acquisition in S1 includes:

[0065] S11, UAV equipment selection and configuration: Select a UAV platform and equip it with multi-dimensional image acquisition equipment, including a visible light image acquisition unit, an infrared thermal imaging unit, and a 3D laser point cloud scanner, ensuring compatibility and data synchronization of each device;

[0066] S12, Bridge Surface and Internal Structure Data Acquisition: By controlling the drone to fly to different positions and angles on the bridge, multimodal data of the bridge surface and internal structure are collected, including datasets of visible light images, infrared thermal imaging data, and 3D laser point cloud data;

[0067] Through the above content, detailed information on the bridge surface and internal structure can be obtained comprehensively and efficiently. By rationally configuring and optimizing equipment, the synchronization and high-precision collection of data are ensured, so that key defect information can be accurately captured under different environments and complex structural conditions. This not only improves the coverage and accuracy of data collection, but also greatly reduces the risks and costs of manual inspections, and improves the automation and intelligence level of bridge health monitoring.

[0068] Multimodal data preprocessing in S2 includes:

[0069] S21, Spatiotemporal Data Synchronization Analysis: Analyzes the temporal and spatial dimensions of multimodal data. By calculating the timestamp and spatial coordinates of each modal data, it identifies synchronization deviations between different data sources and ensures that all data are consistent in time and space. This is expressed as:

[0070] T aligned =T visible +ΔT;

[0071] Among them, T aligned is the timestamp after alignment, T visible is the timestamp of the visible light image data, ΔT is the adjustment deviation time;

[0072] S22, application of spatiotemporal enhancement algorithm: enhance the multimodal data through spatiotemporal synchronization enhancement algorithm to improve the spatiotemporal alignment accuracy of each modal data. Assume that the input multimodal data is D = [D1, D2, ..., D n ], i=[1,2,...,n], where each D i Represents a modal data (visible light, infrared, laser point cloud), the spatiotemporal enhancement algorithm generates an enhanced dataset D by adjusting the temporal and spatial relationship between the modalities. aug , expressed as:

[0073] D aug =f(D,T aligned ,S aligned );

[0074] Among them, D aug is the enhanced multimodal dataset, f(·) is the spatiotemporal enhancement function, T aligned is the timestamp after alignment, S aligned is the spatial coordinate after alignment;

[0075] The spatiotemporal enhancement function f(·) is expressed as:

[0076]

[0077] Among them, D aug is the multimodal dataset after spatiotemporal enhancement, w i is the modal data D i The weight coefficient of For the modal data D i Processing functions for time-space alignment, including time and space synchronization operations, and Mode D i The aligned timestamps and spatial coordinates of , n is the number of modalities (visible light image, infrared thermal imaging data, 3D laser point cloud data);

[0078]

[0079] in, It is an interpolation function used to perform temporal and spatial compensation and interpolation on data;

[0080]

[0081] Among them, D i is the modal data D iThe value at the current moment, T current is the current timestamp, T aligned is the alignment timestamp of the modal data, ΔT is the time period length, D i ' is the modal data D i The value at the interpolated moment;

[0082] S23, generate a spatiotemporal aligned dataset: based on the generated enhanced dataset D aug , generate a spatiotemporal aligned augmented dataset D allgned , expressed as:

[0083]

[0084] in, Represents each modal data and its corresponding timestamp and spatial coordinates, i = [1, 2, ..., n];

[0085] S24, data quality assessment: The spatiotemporal alignment accuracy of the dataset is evaluated by calculating the mean alignment error (MAE), which is expressed as:

[0086]

[0087] Among them, MAE is the mean alignment error, is the timestamp after alignment, is the reference timestamp, N is the number of data points;

[0088] Through the above content, the consistency of various types of data in time and space is ensured. Through weighted interpolation and spatiotemporal compensation, the accuracy and reliability of data fusion are improved, so that different modal data can work together accurately at the same time and space point. This not only improves the integrity and information complementarity of the data, but also provides higher-quality input data for subsequent deep learning models, thereby enhancing the accuracy and robustness of defect identification, and further promoting the intelligent and automated development of bridge health monitoring.

[0089] Feature fusion and extraction in S3 include:

[0090] S31, inputting the spatiotemporally aligned augmented dataset: inputting the spatiotemporally aligned augmented dataset into the multimodal feature fusion network;

[0091] S32, feature extraction using cross-scale convolution kernel groups: Through the cross-scale convolution kernel group in the multimodal feature fusion network, defect feature maps F of different scales and dimensions are extracted. i , each modal data D i The goal of the cross-scale convolution kernel group is to extract multi-level features from each modal data and capture the multi-scale information of the defect by performing convolution operations with multiple convolution kernels of different sizes. It can be expressed as:

[0092]

[0093] Among them, F i From mode D i The extracted feature map, W k is the kth convolution kernel, the convolution kernel size is k×k, D i is the input modal data of the i-th type, K is the total number of convolution kernels (the number of cross-scale convolution kernel groups);

[0094] S33, fusion of different modal features: the feature maps F1, F2, ..., F from different modalities are combined n Perform fusion to generate a multimodal fusion feature map F fusion , expressed as:

[0095]

[0096] Among them, F fusion is the fused feature map, w i ' is the weight coefficient of mode i, which indicates the contribution of the mode to the fusion feature map, F i is the feature map of the i-th mode;

[0097] Through the above content, the advantageous information of each modality can be effectively integrated, the ability to express defect characteristics can be improved, and multi-level features can be extracted through cross-scale convolution kernel groups. Combined with the weighted averaging method, it ensures that the features of different modalities are reasonably weighted according to their importance during the fusion process. This not only enhances the complementarity of the modal data, but also improves the accuracy and robustness of the fused feature map.

[0098] The adaptive optimization feature maps in S4 include:

[0099] S41, real-time environmental parameter collection: The current environmental parameters E, including temperature, humidity, and light intensity, are collected in real time through environmental sensors and are expressed as:

[0100] E=[E1,E2,...,E m ];

[0101] Among them, E i is the i-th environmental parameter, i=[1,2,...,m];

[0102] S42, dynamic weight allocation mechanism design: According to the real-time environment parameter E, the weight of each layer of the network is adjusted through an adaptive mechanism, which is expressed as:

[0103]

[0104] Among them, w i,t' is the dynamic adjustment weight of the i-th layer at time t, α i is the basic weight coefficient of the layer, β i is the environmental parameter E i The influence coefficient of the layer weight, E i is the i-th real-time environmental parameter, max(E) is the maximum value of all environmental parameters, which is used for normalization;

[0105] S43, optimize the weight connection of the fusion feature map: dynamically optimize the connection weights of each network layer according to the real-time environment parameters, ensure that the output of each layer adjusts its importance according to the current environment, and output the adaptively optimized fusion feature map F opt , expressed as:

[0106]

[0107] Among them, F opt is the fusion feature map after adaptive optimization, F i is the feature map of the i-th layer;

[0108] Through the above content, the connection weights between network layers are adaptively adjusted according to real-time environmental parameters, so that the model can flexibly respond to the impact of different environmental conditions on data quality and feature performance. By optimizing the weights of the fused feature maps in real time, the model's adaptability to various environmental changes can be effectively improved, and recognition accuracy and robustness can be enhanced. This not only improves the performance of the model in complex environments, but also ensures refined adjustments during feature extraction and fusion, making defect identification more accurate.

[0109] Defect location and type identification in S5 include:

[0110] S51, defect spatial coordinate positioning: The adaptively optimized fusion feature map is input into the bidirectional attention positioning mechanism to locate the spatial coordinates of the defect. Through the attention mechanism, the model can focus on the defect area and suppress irrelevant information, thereby effectively identifying and calibrating the defect location;

[0111] S52, defect type identification and probability distribution output: Based on the spatial coordinates of the located defects, the bidirectional attention mechanism identifies the type of each defect and outputs the probability distribution of the defect type;

[0112] Through the above content, the spatial coordinates of the defects are accurately located and the defect types are classified and identified. Through the attention mechanism, the model can focus on the defect area and effectively eliminate interference information, thereby improving positioning accuracy and classification accuracy. It not only improves the reliability of defect detection, but also provides probability distribution for different types of defects, providing strong support for subsequent bridge maintenance and repair decisions, and enhancing the practicality and accuracy of the intelligent inspection system.

[0113] The defect spatial coordinate positioning in S51 includes:

[0114] S511, Bidirectional Attention Mechanism Calculation: In the bidirectional attention mechanism, the adaptively optimized fusion feature map F is forward propagated. opt Process and calculate the importance weight A of each position i,j , and adjust and strengthen the weight of the defect area through back propagation. The bidirectional attention mechanism is expressed as:

[0115]

[0116] Among them, A i,j is the feature map F opt The attention weight of the i-th row and j-th column in the , represents the attention degree of the position to the defect, F opt (i,j) is the fusion feature map F opt The eigenvalue of the i-th row and j-th column, H is the height or number of rows of the feature map, F opt (i,k) is the fusion feature map F opt The eigenvalue at row i and column k;

[0117] S512, defect space coordinate calculation: according to the calculated attention weight A i,j , select the area with the highest weight as the spatial coordinate C of the defect, expressed as:

[0118] C = argmax i,j A i,j ;

[0119] Where C is the spatial coordinate of the defect, indicating the image location with the highest attention weight;

[0120] Through the above content, it is possible to accurately focus on the defect area and suppress irrelevant information, thereby significantly improving the accuracy of defect positioning. Through the adaptively optimized fusion feature map, it is possible to fully utilize the data advantages of different modalities and perform precise spatial coordinate positioning. This not only improves the recognition accuracy of the defect area, but also can be dynamically adjusted according to the attention weight of the feature map, ensuring stable and efficient detection and positioning of defects in different environments.

[0121] Defect type identification and probability distribution output in S52 include:

[0122] S521, extract the defect area based on the spatial coordinates: based on the acquired spatial coordinates of the defect C = (x c ,y c ), from the adaptively optimized fusion feature map F opt Extract the feature map F corresponding to the defect area region , expressed as:

[0123] F region =Extract(F opt ,C);

[0124] Among them, F region is the extracted defect area feature map, C=(x c ,y c ) is the spatial coordinate of the defect, and Extract represents the operation of extracting the defect area from the feature map;

[0125] S522, defect type identification: extract the defect area feature map F region Input into the bidirectional attention mechanism, perform type recognition based on the characteristics of the defect area, and calculate the probability P of each defect type through forward propagation t , expressed as:

[0126]

[0127] Among them, P t is the probability of the t-th defect type, W t is the classification weight of the t-th defect type, F region is the extracted defect area feature map, T is the total number of defect types, W k Classification weights for other defect types;

[0128] S523, output defect type probability distribution: according to the calculated defect type probabilities P1, P2, ..., P T , output the type of each defect area and its corresponding probability distribution, expressed as:

[0129] P type =[P1,P2,...,P T ];

[0130] Among them, P type is the probability distribution of defect types, indicating the probability of each type of defect;

[0131] Through the above content, it is possible to accurately identify the type of defect based on the located defect area and output the probability distribution of each defect type. By extracting the features of the defect area and combining it with the attention mechanism, it is possible to focus on important features, suppress noise, and improve classification accuracy. It can not only accurately identify the defect type, but also provide a reliable probability distribution for each type, helping decision makers to assess the severity of the defect and the repair priority, thereby optimizing bridge maintenance strategies.

[0132] Defect analysis and report generation in S6 include:

[0133] S61, defect evolution trend analysis: Based on historical multimodal data and the spatial coordinates and type identification results of the current defect, the evolution trend of the defect is analyzed, and the future development trend of the defect is predicted through the regression analysis model, which is expressed as:

[0134] e i+1 =α·e i +β·Δe i ;

[0135] Among them, e i+1 is the defect state prediction value at the next time point, α and β are the regression coefficients of the model, Δe i is the change in the current defect state, indicating the change in the defect within the time interval, e i is the state of the defect at the i-th time point;

[0136] S62, generate inspection report: Combine the results of defect location and type identification and defect evolution trend analysis to generate an inspection report including the defect evolution trend;

[0137] The above content can provide more comprehensive and accurate data support for bridge maintenance. Through evolution trend prediction, potential risks and defect expansion can be identified in advance, helping decision makers to take more effective preventive and maintenance measures. At the same time, the detailed defect type and evolution trend analysis in the report can provide a continuous reference for long-term health monitoring of bridges, improve the accuracy and efficiency of inspections, and ultimately optimize the allocation of maintenance resources.

[0138] Generating an inspection report in S62 includes:

[0139] S621, Defect Location and Type: Lists the location coordinates of each defect and the probability distribution of type identification;

[0140] S622, Defect Evolution Trend: Shows the evolution trend of defects at different time points and predicts the future state changes of defects;

[0141] S623, Maintenance Recommendations: Based on the type and evolution trend of defects, make corresponding repair or reinforcement recommendations, including crack repair, corrosion prevention and reinforcement, local reinforcement and replacement, bridge deck or bearing replacement, extended monitoring and tracking, and structural reconstruction and renovation;

[0142] Through the above content, the defect location, probability distribution of type identification, evolution trend and future prediction are listed in detail, and combined with maintenance recommendations, it provides comprehensive and accurate data support for bridge maintenance decisions. By showing the evolution trend of defects, the report can provide early warning of potential structural risks and help managers make timely repair or reinforcement decisions. The maintenance recommendations cover different types of defects and repair plans, ensuring that the most appropriate maintenance measures can be taken in different situations. This report not only improves the intelligence and refinement level of bridge management, but also optimizes resource allocation, extends the service life of bridges, and ensures bridge safety and reliability.

[0143] The present invention encompasses any alternatives, modifications, equivalents, and solutions that fall within the spirit and scope of the present invention. To provide a thorough understanding of the present invention, specific details are described in detail below in connection with the preferred embodiments of the present invention, but those skilled in the art will be able to fully understand the present invention without these detailed descriptions. Furthermore, to avoid unnecessary confusion regarding the essence of the present invention, well-known methods, processes, procedures, components, and circuits have not been described in detail.

[0144] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for automatic identification and inspection of bridge defects based on deep learning, characterized by: The following steps are involved: S1, multimodal data acquisition: The multi-dimensional image acquisition equipment carried by the UAV is used to obtain multimodal data of the bridge surface and internal structure, including visible light images, infrared thermal imaging data, and 3D laser point cloud data; S2, multimodal data preprocessing: a spatiotemporal synchronization enhancement algorithm is used to preprocess the multimodal data to generate a spatiotemporally aligned enhanced dataset; S3, feature fusion and extraction: The spatiotemporally aligned augmented dataset is fed into a multimodal feature fusion network, which extracts defect features of different dimensions through a cross-scale convolution kernel group to generate a fused feature map. S4, adaptive optimization of feature maps: Adaptively optimize the fused feature maps based on a dynamic weight allocation mechanism, and adjust the connection weights between network layers according to real-time environment parameters; S5, defect location and type identification: The optimized fusion feature map is input into the bidirectional attention localization mechanism to locate the spatial coordinates of the defect and simultaneously output the probability distribution of the defect type; specifically, it includes: S51, defect spatial coordinate positioning: the adaptively optimized fusion feature map is input into the bidirectional attention positioning mechanism to locate the spatial coordinates of the defect; S52, defect type identification and probability distribution output: Based on the spatial coordinates of the located defects, the bidirectional attention mechanism identifies the type of each defect and outputs the probability distribution of the defect type. S6, Defect Analysis and Report Generation: Based on the results of defect location and type identification, an inspection report including defect evolution trend analysis is generated.

2. The method for automatic identification and inspection of bridge defects based on deep learning according to claim 1 is characterized in that: The multimodal data acquisition in S1 includes: S11, UAV equipment selection and configuration: Select a UAV platform and equip it with multi-dimensional image acquisition equipment, including a visible light image acquisition unit, an infrared thermal imaging unit, and a 3D laser point cloud scanner; S12, Bridge Surface and Internal Structure Data Collection: By controlling the drone to fly to different positions and angles of the bridge, multimodal data of the bridge surface and internal structure are collected, and a data set including visible light images, infrared thermal imaging data, and 3D laser point cloud data is obtained.

3. The method for automatic identification and inspection of bridge defects based on deep learning according to claim 1 is characterized in that: The multimodal data preprocessing in S2 includes: S21, Spatiotemporal Data Synchronization Analysis: Analyze the temporal and spatial dimensions of multimodal data, and identify synchronization deviations between different data sources by calculating the timestamps and spatial coordinates of each modality; S22, Application of spatiotemporal enhancement algorithm: Enhance multimodal data through spatiotemporal synchronization enhancement algorithm, improve the spatiotemporal alignment accuracy of each modality data, and generate enhanced data sets ; S23, generate a spatiotemporal aligned dataset: based on the generated enhanced dataset , generating a spatiotemporally aligned augmented dataset ; S24, Data quality assessment: Evaluate the spatiotemporal alignment accuracy of the dataset by calculating the average alignment error.

4. The method for automatic identification and inspection of bridge defects based on deep learning according to claim 1 is characterized in that: The feature fusion and extraction in S3 include: S31, inputting the spatiotemporally aligned augmented dataset: inputting the spatiotemporally aligned augmented dataset into the multimodal feature fusion network; S32, feature extraction using cross-scale convolution kernel groups: Defect feature maps of different scales and dimensions are extracted using cross-scale convolution kernel groups in the multimodal feature fusion network. ; S33, Fusion of different modal features: Fusion of feature maps from different modalities Perform fusion to generate multimodal fusion feature maps .

5. The method for automatic identification and inspection of bridge defects based on deep learning according to claim 3 is characterized in that: The adaptive optimization feature map in S4 includes: S41, real-time environmental parameter collection: collect current environmental parameters in real time through environmental sensors , including temperature, humidity, and light intensity; S42, Dynamic Weight Allocation Mechanism Design: Based on Real-time Environment Parameters , adjust the weights of each layer of the network through an adaptive mechanism; S43, optimize the weight connection of the fusion feature map: dynamically optimize the connection weights of each network layer according to the real-time environment parameters, and output the adaptively optimized fusion feature map .

6. The method for automatic identification and inspection of bridge defects based on deep learning according to claim 1 is characterized in that: The defect spatial coordinate positioning in S51 includes: S511, Bidirectional Attention Mechanism Calculation: In the bidirectional attention mechanism, the fusion feature map after adaptive optimization is propagated forward. Process and calculate the importance weight of each position , and adjust and strengthen the weights of the defective areas through back propagation; S512, defect space coordinate calculation: according to the calculated attention weight , select the area with the highest weight as the spatial coordinate of the defect .

7. The method for automatic identification and inspection of bridge defects based on deep learning according to claim 6 is characterized in that: The defect type identification and probability distribution output in S52 include: S521, extracting defect areas based on spatial coordinates: Based on the acquired spatial coordinates of the defect , from the fusion feature map after adaptive optimization Extract the feature map corresponding to the defect area ; S522, defect type identification: extract the defect area feature map Input into the bidirectional attention mechanism, perform type recognition based on the characteristics of the defect area, and calculate the probability of each defect type through forward propagation ; S523, output defect type probability distribution: according to the calculated defect type probability , output the type of each defect area and its corresponding probability distribution.

8. The method for automatic identification and inspection of bridge defects based on deep learning according to claim 7 is characterized in that: The defect analysis and report generation in S6 includes: S61, Defect Evolution Trend Analysis: Based on historical multimodal data and the spatial coordinates and type identification results of the current defect, the defect evolution trend is analyzed and the future development trend of the defect is predicted using a regression analysis model; S62, generating an inspection report: combining the results of defect location and type identification and defect evolution trend analysis, generating an inspection report including the defect evolution trend.

9. The method for automatic identification and inspection of bridge defects based on deep learning according to claim 8 is characterized in that: The generating of the inspection report in S62 includes: S621, Defect Location and Type: Lists the location coordinates of each defect and the probability distribution of type identification; S622, Defect Evolution Trend: Shows the evolution trend of defects at different time points and predicts the future state changes of defects; S623, Maintenance Recommendations: Based on the type and evolution trend of the defects, make corresponding maintenance or reinforcement recommendations, including crack repair, corrosion prevention and reinforcement, local reinforcement and replacement, bridge deck or bearing replacement, extended monitoring and tracking, and structural reconstruction and renovation.

Images