Multi-modal information fusion steel-UHPC combined bridge dynamic maintenance system and method

Abstract

The invention discloses a multi-modal information fused steel-UHPC combined bridge dynamic maintenance system and method, and relates to the technical field of crack detection, and the method comprises the steps: obtaining an optimized sensor network based on a sensor layout optimization model, a sensor collaborative layout optimization model and a sensor number dynamic adjustment method, and monitoring multi-modal information; based on a pre-trained damage category identification model, inputting multi-modal information, and outputting a damage category; on the basis of the damage category, utilizing a pre-trained bridge damage evolution model to predict bridge damage characteristics of adjacent time steps, and calculating a damage expansion rate; performing local grid dynamic division on the bridge based on the optimized sensor network, obtaining a damage positioning result, calculating the degradation degree of the bridge structure, and evaluating the residual life of the bridge in combination with the damage expansion rate; a maintenance strategy knowledge base is constructed, a maintenance strategy set is generated according to the bridge damage category, the remaining life and the bridge structure deterioration degree, and the precision of bridge damage detection and maintenance prevention is improved.

Description

Technical Field

[0001] The present application relates to the field of crack detection technology, and in particular to a dynamic maintenance system and method for a steel-UHPC composite bridge using multi-modal information fusion. Background Art

[0002] Steel-UHPC composite bridges combine the high strength of steel structures and the excellent durability of UHPC. The two materials work synergistically through precise interface connections, and have high load-bearing capacity, fatigue resistance and long service life. However, during long-term service, steel-UHPC composite bridges may be affected by multiple factors such as vehicle loads, fatigue damage, environmental erosion and material aging, resulting in performance degradation and threatening structural safety.

[0003] The patent application with publication number CN117952601A discloses a bridge health big data intelligent management and maintenance system, including a detection module, including a detector, for collecting bridge data and environmental data; the bridge data includes bridge structure data and bridge characteristic data; the bridge structure data includes spatial information of the bridge, and the bridge characteristic data includes bridge surface information; a mobile module includes a mobile module and a fixed module; wherein, the detection device is arranged on the back side of the bridge through the fixed module, and the detection device is movably arranged on the back side of the bridge through the mobile module and the fixed module, and the detection device is used to detect cracks on the back side of the bridge.

[0004] Existing methods mostly rely on single crack data for analysis and fail to combine multimodal information, making it difficult to accurately capture the damage of multi-material systems of bridges. Summary of the invention

[0005] The present application aims to solve at least one of the technical problems in the related art to a certain extent. To this end, one purpose of the present application is to propose a dynamic maintenance system and method for a steel-UHPC composite bridge using multimodal information fusion, thereby improving the accuracy of bridge damage detection and maintenance prevention.

[0006] One aspect of the present application provides a dynamic maintenance method for a steel-UHPC composite bridge using multi-modal information fusion, comprising:

[0007] Step S100: Based on the sensor deployment optimization model, the sensor collaborative deployment optimization model and the sensor quantity dynamic adjustment method, an optimized sensor network is obtained to monitor multimodal information;

[0008] Step S200: Based on the pre-trained damage classification recognition model, multi-modal information is input and damage classification is output;

[0009] Step S300: Based on the damage category, the pre-trained bridge damage evolution model is used to predict the bridge damage characteristics of adjacent time steps and calculate the damage propagation rate;

[0010] Step S400: dynamically divide the local grid of the bridge based on the optimized sensor network, obtain damage location results and calculate the degree of bridge structure degradation, and evaluate the remaining life of the bridge in combination with the damage extension rate;

[0011] Step S500: construct a maintenance strategy knowledge base, and generate a maintenance strategy set according to the bridge damage category, remaining life and bridge structure deterioration degree;

[0012] The specific method for obtaining an optimized sensor network and monitoring multimodal information based on the sensor deployment optimization model, the sensor collaborative deployment optimization model and the sensor quantity dynamic adjustment method is as follows:

[0013] Step S110: defining a damage location accuracy requirement matrix;

[0014] Step S120: defining a sensor deployment decision matrix, which is used to represent the number of sensors deployed in each area;

[0015] Step S130: Taking minimizing the sensor deployment cost as the optimization objective function and the number of sensors as the optimization variable, defining the positioning accuracy contribution constraint of the sensors, building a sensor deployment optimization model, and solving the initial optimal number of sensors in each area through the optimization algorithm;

[0016] Step S140: defining a damage localization result matrix, which is used to represent the damage density obtained by each damage localization in each area;

[0017] Step S150: designing a method for dynamically adjusting the number of sensors, and updating the number of sensors according to the relationship between the damage density and a preset damage density threshold upper limit and damage density threshold lower limit;

[0018] Step S160: defining a heterogeneous sensor collaborative decision matrix, which is used to represent the number of sensor pairs of different types deployed at the same time;

[0019] Step S170: Taking maximizing the contribution of sensor collaborative positioning as the optimization objective function, taking the number of sensor pairs deployed simultaneously as the optimization variable, defining the sensor pair number constraint, establishing a sensor collaborative deployment optimization model, and obtaining the optimal number of sensor pairs through an optimization algorithm;

[0020] Step S180: based on the optimal number of sensors, the method for dynamically adjusting the number of sensors, and the optimal number of sensor pairs, an optimized sensor network is obtained, and multimodal information is collected based on the sensor network; the multimodal information includes acoustic emission signals, vibration signals, strain signals, and image data;

[0021] The method for dynamically adjusting the number of sensors is as follows: dynamically adjusting the sensor deployment decision matrix according to the damage location result matrix, presetting the upper limit of the damage density threshold and the lower limit of the damage density threshold, and for each area, when the damage density of the sensors in the area is greater than the upper limit of the damage density threshold, updating the number of sensors in the area, and the updated number of sensors is the sum of the current number of sensors in the area and the sensor number adjustment step; when the damage density of the sensors in the area is less than the lower limit of the damage density threshold, calculating the difference between the current number of sensors in the area and the sensor number adjustment step, comparing the difference with the lower limit of the sensor number, and selecting the maximum value between the two as the updated number of sensors;

[0022] The specific method of the damage classification recognition model based on pre-training, inputting multimodal information, and outputting damage classification is as follows:

[0023] Step S210: constructing a damage category recognition model, wherein the damage category recognition model includes a feature extraction module, an attention fusion module, a deep belief network module, and a loss classification module;

[0024] Step S220: preprocessing the multimodal information and extracting multimodal features based on a feature extraction module;

[0025] Step S230: using the attention mechanism to calculate the weights of the multimodal features, fusing the weighted multimodal features based on the attention fusion module to construct a multimodal feature vector;

[0026] Step S240: Input the multimodal feature vector into the deep belief network module to learn the high-level feature representation h x ;

[0027] Step S250: input the high-level feature representation into the loss classification module, use the Softmax classifier to predict the probability of damage category for the high-level feature representation, output the probability of belonging to each damage category, use the cross entropy loss function to calculate the difference between the predicted damage category and the actual damage category, and optimize the model parameters through the back propagation algorithm to obtain a trained damage category recognition model;

[0028] Step S260: using the trained damage category recognition model to predict the probability that the current multimodal information belongs to each damage category, and taking the damage category with the largest probability value as the damage category of the current multimodal information; the damage categories include crack damage, steel bar corrosion damage, concrete carbonization damage, and fatigue accumulation damage;

[0029] The specific method of predicting the bridge damage characteristics of adjacent time steps based on the damage category and calculating the damage extension rate using the pre-trained bridge damage evolution model is as follows:

[0030] Step S310: measuring bridge damage data corresponding to the multimodal information, and grouping the bridge damage data based on different damage categories output;

[0031] Step S320: for each damage category, extracting its damage feature evolution sequence that changes with time from the bridge damage data; the damage features include crack width, crack length and damage area;

[0032] Step S330: construct a bridge damage evolution model, including an input layer, an LSTM hidden layer and an output layer. The input layer is used to receive the damage feature evolution sequence, input a damage feature at each time step, use the LSTM hidden layer to encode the damage feature of the current time step, and map the output of the LSTM hidden layer to the damage feature prediction value of the next time step;

[0033] Step S340: designing a loss function based on the mean square error and the Paris law regularization term, taking the value of minimizing the loss function as the training goal, and training the bridge damage evolution model;

[0034] Step S350: Use the trained bridge damage evolution model to predict damage characteristics, obtain the damage characteristic change Δx of adjacent time steps, and calculate the damage extension rate v x ;

[0035] The loss function is designed based on the mean square error and the Paris law regularization term, and the value of the minimization loss function is used as the training goal. The specific method for training the bridge damage evolution model is:

[0036] Step S341: Obtain damage feature evolution sequence samples and divide them into subsequences of fixed length Z. For each subsequence, use the damage features of the first Z-1 time steps as input, use the damage features of the last time step as prediction targets, and combine the inputs of all subsequences and their corresponding prediction targets into training sample pairs.

[0037] Step S342: using the mean square error to calculate the difference between the true value of the damage feature and the predicted value of the damage feature, introducing the Paris law regularization term, and obtaining the final loss function Loss based on the weighted sum of the mean square error and the Paris law regularization term. Taking the value of the minimization loss function as the training goal, the model parameters are optimized to obtain a trained bridge damage evolution model;

[0038] The specific method of dynamically dividing the local grid of the bridge based on the optimized sensor network, obtaining the damage location result and calculating the degree of deterioration of the bridge structure, and evaluating the remaining life of the bridge in combination with the damage extension rate is as follows:

[0039] Step S410: obtaining the geometric topological relationship of the bridge structure and obtaining the grid size Δp of the optimized sensor network;

[0040] Step S420: dynamically adjusting the local grid size according to the intensity of the acoustic emission signal in the multimodal information. When the intensity of the acoustic emission signal of the grid unit in the optimized sensor network is greater than the upper limit of the intensity threshold, the local grid size Δp' is adjusted to half of the original grid size; when the intensity of the acoustic emission signal of the grid unit in the optimized sensor network is less than the lower limit of the intensity threshold, the local grid size Δp' is adjusted to twice the original grid size;

[0041] Step S430: define the acoustic wave propagation path compensation term Δs, calculate the difference between the recorded acoustic emission signal arrival time and the acoustic wave propagation path compensation term, and obtain the corrected acoustic emission signal arrival time s';

[0042] Step S440: Preliminarily locate the acoustic emission signal on the grid size Δp to obtain a suspected damage area, and obtain a damage location result in a local grid of the suspected damage area;

[0043] Step S450: Count the number of damage points according to the damage location results and calculate the damage density l af , according to the time delay and energy attenuation of the acoustic emission signal propagating in the bridge structure, the degree of deterioration of the bridge structure is calculated;

[0044] Step S460: Calculate the remaining life Lr of the bridge structure according to the bridge structure degradation degree and damage extension rate;

[0045] The specific method for preliminarily locating the acoustic emission signal on the grid size Δp to obtain the suspected damage area is:

[0046] Step S441: Assume that the position of the sound source is (xi, yi, zi), the position of the ni-th sensor is (xni, yni, zni), and the sound wave propagation speed is vi;

[0047] Step S442: Calculate the distance dni from the sound source to the ni-th sensor, and calculate the time tni when the acoustic emission signal reaches the ni-th sensor;

[0048] Step S443: for any two sensors ni, nj, calculate the time difference Δtij between them receiving the acoustic emission signals;

[0049] Step S444: Substitute the distance into the calculation formula of the time difference and obtain the time difference equation group, solve the time difference equation group, obtain the sound source position as the preliminary positioning result, and obtain the suspected damage area;

[0050] The specific method of constructing the maintenance strategy knowledge base and generating a maintenance strategy set according to the bridge damage category, the remaining life and the degree of bridge structure deterioration is as follows:

[0051] Step S510: collecting expert knowledge in the field of bridge maintenance, including the corresponding relationship between damage categories, bridge structure degradation degree, remaining life and maintenance strategies, and constructing a maintenance strategy knowledge base;

[0052] Step S520: Taking the bridge damage category, remaining life and bridge structure deterioration degree as input, matching feasible maintenance strategies in the maintenance strategy knowledge base, generating a maintenance strategy set, and sending it to maintenance personnel.

[0053] One aspect of the present application provides a dynamic maintenance system for a steel-UHPC composite bridge using multi-modal information fusion, comprising:

[0054] The multimodal information acquisition module obtains the optimized sensor network and monitors the multimodal information based on the sensor layout optimization model, sensor collaborative layout optimization model and sensor quantity dynamic adjustment method;

[0055] The damage category output module, based on the pre-trained damage category recognition model, inputs multimodal information and outputs damage categories;

[0056] The damage extension calculation module uses the pre-trained bridge damage evolution model to predict the bridge damage characteristics of adjacent time steps based on the damage category and calculate the damage extension rate;

[0057] The remaining life assessment module dynamically divides the local grid of the bridge based on the optimized sensor network, obtains the damage location results and calculates the degree of deterioration of the bridge structure, and evaluates the remaining life of the bridge in combination with the damage extension rate;

[0058] The maintenance strategy generation module is used to build a maintenance strategy knowledge base and generate a set of maintenance strategies based on bridge damage categories, remaining life, and bridge structure deterioration degree.

[0059] One aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the processor executes the steps in the dynamic maintenance method of a steel-UHPC composite bridge using multimodal information fusion.

[0060] The multi-modal information fusion steel-UHPC composite bridge dynamic maintenance system and method proposed in this application have the following advantages over the prior art:

[0061] This application comprehensively considers factors such as damage location accuracy requirements and sensor deployment costs, constructs a sensor deployment optimization model, realizes quantitative optimization of sensor deployment, introduces a sensor collaborative deployment optimization model, considers the synergistic effect of heterogeneous sensors, further improves the monitoring performance of the sensor network, and adaptively adjusts the number of sensors according to real-time damage monitoring results to achieve dynamic optimization of the sensor network.

[0062] This application constructs a damage category recognition model that includes modules such as feature extraction, attention fusion, and deep belief network. It can effectively process and fuse multimodal monitoring data, and use the attention mechanism to adaptively assign different modal features and weights, highlight the role of key modalities, and improve the accuracy of damage identification.

[0063] This application constructs a special damage evolution model for different damage categories, fully considers the evolution characteristics and mechanisms of different damage modes, and uses LSTM network to model the damage evolution process, which can capture the long-term dependency of damage development and improve the accuracy of evolution prediction. It introduces the Paris law regularization term in the loss function, incorporates the physical mechanism of damage expansion, and improves the physical rationality of damage evolution prediction.

[0064] This application dynamically adjusts the local grid size according to the intensity of the acoustic emission signal, improves the accuracy of damage location while ensuring calculation efficiency, comprehensively considers the degree of bridge structure deterioration and damage expansion rate, gives an overall assessment of the remaining life of the bridge, achieves accurate positioning and quantitative assessment of bridge damage at multiple scales and levels, and predicts the remaining life of the bridge based on the degree of damage and evolution trend, providing a comprehensive technical means for performance degradation analysis of multi-material bridge systems.

[0065] This application considers the damage characteristics, deterioration laws and life expectancy of the multi-material system of bridges, and provides a comprehensive and systematic maintenance strategy to effectively improve the scientificity and effectiveness of bridge maintenance and effectively extend the service life of bridges.

[0066] This application is based on multimodal monitoring data, integrating advanced data analysis, damage identification, life prediction and other methods, supplemented by expert knowledge, to form a complete health monitoring and life management solution for multi-material bridge systems. Compared with the existing method that relies solely on crack data, this application can comprehensively and accurately analyze bridge damage, grasp its evolution law, evaluate structural performance, and provide targeted maintenance measures. It has significant technical advantages and application prospects in improving bridge safety and extending bridge life. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 A method flow chart of the dynamic maintenance method of steel-UHPC composite bridge with multi-modal information fusion provided in this application;

[0068] Figure 2 A flow chart of the optimization method for optimizing a sensor network provided in this application;

[0069] Figure 3 Flow chart of the remaining life assessment method of bridge structure provided for this application;

[0070] Figure 4 Functional module diagram of the dynamic maintenance system of steel-UHPC composite bridge with multimodal information fusion provided in this application. DETAILED DESCRIPTION

[0071] In order to better understand the present application, various aspects of the present application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present application and do not limit the scope of the present application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0072] In the accompanying drawings, the size, dimensions and shapes of the elements have been slightly adjusted for ease of illustration. The drawings are for illustration only and are not strictly drawn to scale. As used herein, the terms "substantially", "approximately" and similar terms are used as terms of approximation, not as terms of degree, and are intended to illustrate the inherent deviations in measurements or calculations that will be recognized by those of ordinary skill in the art. In addition, in this application, the order in which the steps are described does not necessarily represent the order in which these processes occur in actual operation, unless otherwise specified or can be derived from the context.

[0073] It should also be understood that expressions such as "including", "comprising", "having", "containing" and / or "comprising" are open rather than closed expressions in this specification, which indicate the presence of the stated features, elements and / or components, but do not exclude the presence of one or more other features, elements, components and / or combinations thereof. In addition, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features rather than just the individual elements in the list. In addition, when describing embodiments of the present application, "may" is used to mean "one or more embodiments of the present application". And, the term "exemplary" is intended to refer to an example or illustration.

[0074] Unless otherwise specified, all words (including engineering terms and scientific and technological terms) used in this article have the same meaning as those commonly understood by ordinary technicians in the field to which this application belongs. It should also be understood that, unless clearly stated in this application, words defined in common dictionaries should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense.

[0075] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0076] Example 1

[0077] like Figure 1 As shown, the dynamic maintenance method of the steel-UHPC composite bridge with multi-modal information fusion provided in this application includes:

[0078] Step S100: Based on the sensor deployment optimization model, the sensor collaborative deployment optimization model and the sensor quantity dynamic adjustment method, an optimized sensor network is obtained to monitor multimodal information;

[0079] The specific method for obtaining an optimized sensor network and monitoring multimodal information based on the sensor deployment optimization model, the sensor collaborative deployment optimization model and the sensor quantity dynamic adjustment method is as follows:

[0080] Step S110: defining a damage location accuracy requirement matrix;

[0081] The damage location accuracy requirement matrix C = [c ab ]; where c ab Indicates the positioning accuracy requirement of the ath area for the bth sensor;

[0082] Step S120: defining a sensor deployment decision matrix, which is used to represent the number of sensors deployed in each area;

[0083] The sensor placement decision matrix D = [d ab ]; where d ab represents the number of sensors of type b deployed in the ath area;

[0084] Step S130: Taking minimizing the sensor deployment cost as the optimization objective function and the number of sensors as the optimization variable, defining the positioning accuracy contribution constraint of the sensors, building a sensor deployment optimization model, and solving the initial optimal number of sensors in each area through the optimization algorithm;

[0085] The calculation formula for minimizing the sensor deployment cost as the optimization objective function is: Among them, cbab is the cost of deploying the bth type of sensor in the ath area, Na is the number of areas divided by the bridge, and Nb is the number of sensor types;

[0086] The calculation formula of the positioning accuracy contribution constraint of the sensor is: Among them, f ab represents the positioning accuracy contribution of the b-th sensor in the a-th region;

[0087] The calculation formula of the positioning accuracy contribution is: Among them, N total represents the total number of sensors in the ath region, Indicates the i-th b The sensor and the i g The distance between the sensors;

[0088] Step S140: defining a damage localization result matrix, which is used to represent the damage density obtained by each damage localization in each area;

[0089] The damage location result matrix L = [l af ]; where l af It represents the damage density of the ath region at the fth location;

[0090] The calculation formula of the damage density is: Among them, n a is the number of damage points detected in the ath region, S a is the area of ​​the ath region;

[0091] The method for calculating the number of damage points is: obtaining acoustic emission signals according to sensors, locating damage points, and counting the number of damage points;

[0092] Step S150: designing a method for dynamically adjusting the number of sensors, and updating the number of sensors according to the relationship between the damage density and a preset damage density threshold upper limit and damage density threshold lower limit;

[0093] The method for dynamically adjusting the number of sensors is as follows: dynamically adjusting the sensor deployment decision matrix according to the damage location result matrix, presetting the upper limit of the damage density threshold and the lower limit of the damage density threshold, and for each area, when the damage density of the sensors in the area is greater than the upper limit of the damage density threshold, updating the number of sensors in the area, and the updated number of sensors is the sum of the current number of sensors in the area and the sensor number adjustment step; when the damage density of the sensors in the area is less than the lower limit of the damage density threshold, calculating the difference between the current number of sensors in the area and the sensor number adjustment step, comparing the difference with the lower limit of the sensor number, and selecting the maximum value between the two as the updated number of sensors;

[0094] The updated number of sensors d ab,new The calculation formula is: in, and are the upper and lower limits of the damage density threshold, respectively. Δd is the sensor quantity adjustment step size. min is the lower limit of the number of sensors, d ab,old is the number of sensors before updating;

[0095] The sensor deployment optimization model determines the starting point of sensor deployment to meet the initial damage location accuracy requirements and cost constraints; step S150 dynamically adjusts the sensor deployment according to the actual damage location results during the monitoring process to adapt to changes in the damage state of the bridge. This step realizes the adaptability of the sensor network, and can increase the number of sensors in damage-intensive areas and reduce the number of sensors in damage-sparse areas, thereby reducing costs while meeting the positioning accuracy requirements.

[0096] Step S160: defining a heterogeneous sensor collaborative decision matrix, which is used to represent the number of sensor pairs of different types deployed at the same time;

[0097] The heterogeneous sensor collaborative decision matrix Y = [y abg ]; where y abg represents the number of sensor pairs where both the bth and gth sensors are deployed in the ath region;

[0098] Step S170: Taking maximizing the contribution of sensor collaborative positioning as the optimization objective function, taking the number of sensor pairs deployed simultaneously as the optimization variable, defining the sensor pair number constraint, establishing a sensor collaborative deployment optimization model, and obtaining the optimal number of sensor pairs through an optimization algorithm;

[0099] The calculation formula for maximizing the sensor cooperative positioning contribution as the optimization objective function is: Among them, eabg represents the collaborative positioning accuracy contribution of the a-th regional sensor pair (b, g), and (b, g) represents the sensor pair consisting of the b-th and g-th sensors;

[0100] The calculation formula of the collaborative positioning accuracy contribution is: Among them, jl (b,g) represents the distance between the sensor pair (b, g), Δt (b,g) It represents the time difference when the sensor pair (b, g) detects the same damage point;

[0101] The calculation formula for the sensor pair quantity constraint is:

[0102] Step S180: based on the optimal number of sensors, the method for dynamically adjusting the number of sensors, and the optimal number of sensor pairs, an optimized sensor network is obtained, and multimodal information is collected based on the optimized sensor network; the multimodal information includes acoustic emission signals, vibration signals, strain signals, and image data;

[0103] Each grid unit in the optimized sensor network represents a possible sensor deployment location, and the sensor network is represented by a grid matrix, each element of the grid matrix corresponds to a grid unit, and the value of the element indicates whether a sensor is deployed at the location, 1 indicates deployment, and 0 indicates no deployment;

[0104] Figure 2 A flow chart of the optimization method for optimizing a sensor network provided in this application;

[0105] The above steps can comprehensively monitor the status of the multi-material system of the bridge through an optimized multimodal sensor network, overcome the limitations of single damage data, and lay the foundation for accurate damage analysis.

[0106] Step S200: Based on the pre-trained damage classification recognition model, multi-modal information is input and damage classification is output;

[0107] The specific method of the damage category recognition model based on pre-training, inputting multimodal information, and outputting damage categories is:

[0108] Step S210: constructing a damage category recognition model, wherein the damage category recognition model includes a feature extraction module, an attention fusion module, a deep belief network module, and a loss classification module;

[0109] The feature extraction module extracts features from multi-source data such as acoustic emission, strain, vibration and image to obtain feature vectors of different modalities; the attention fusion module uses the attention mechanism to adaptively learn the weights of different modal features and fuses the weighted features; the deep belief network module uses the pre-trained deep belief network to perform high-level feature learning on the fused features; the loss classification module uses the Softmax classifier to perform probability prediction of damage categories on high-level features.

[0110] Step S220: preprocessing the multimodal information and extracting multimodal features based on a feature extraction module;

[0111] The specific method for preprocessing multimodal information and extracting multimodal features is as follows:

[0112] Perform short-time Fourier transform on the acoustic emission signal to extract the time-frequency domain characteristics of the acoustic emission signal, including the frequency center and frequency peak, and extract the time-domain characteristics of the acoustic emission signal, including energy and rise time;

[0113] Perform wavelet transform on the vibration signal and extract the wavelet coefficients as the time-frequency domain features of the vibration signal. At the same time, extract the time-domain features of the vibration signal, including the mean and peak factor.

[0114] Extract statistical features of the strain signal, including the mean and variance of the strain signal, and extract frequency domain features of the strain signal, including the frequency amplitude and frequency phase after Fourier transform;

[0115] Extract features from image data to obtain image semantic features;

[0116] The multimodal features are composed of acoustic emission signal time-frequency domain features, acoustic emission signal time-domain features, vibration signal time-frequency domain features, vibration signal time-domain features, statistical features, strain signal frequency domain features and image semantic features;

[0117] Step S230: using the attention mechanism to calculate the weights of the multimodal features, fusing the weighted multimodal features based on the attention fusion module to construct a multimodal feature vector;

[0118] The calculation formula of the weight of the multimodal feature is: Among them, f(·) is the attention scoring function, where α i represents the weight of the i-th multimodal feature, x i 、x i' are the i-th and i'-th multimodal features respectively, and M is the number of modes of the multimodal feature;

[0119] The multimodal feature vector is a high-dimensional feature vector, which performs weighted summation on features of different modalities, wherein the weights are adaptively learned by the attention mechanism of the damage category recognition model, and the features of different modalities are mapped to a common feature space to obtain a fused multimodal feature vector.

[0120] The attention scoring function is a multi-layer perceptron, which is used to calculate the importance score of the multimodal feature, input the multimodal feature into the multi-layer perceptron, output the feature representation, map the feature representation to an importance score through a fully connected layer, and normalize the importance score of the multimodal feature through a softmax function to obtain the attention weight.

[0121] The process of inputting multimodal features into a multilayer perceptron and outputting feature representation is as follows:

[0122] in, is the feature representation of the output of the Lth hidden layer of the multilayer perceptron, W L 、b L is the weight matrix and bias vector of the Lth hidden layer, L is the number of fully connected layers of the multilayer perceptron, and σ(·) is the activation function;

[0123] The feature representation is mapped to an importance score through a fully connected layer as follows: Among them, el i is the importance score of the i-th multimodal feature, v T represents the transpose of the weight vector of the fully connected layer, and b represents the bias term;

[0124] The weight matrix and bias vector of the hidden layer of the multilayer perceptron, the weight vector and bias term of the fully connected layer are model parameters, which are learned and optimized during the training process.

[0125] Step S240: Input the multimodal feature vector into the deep belief network module to learn the high-level feature representation h x ;

[0126] Step S250: input the high-level feature representation into the loss classification module, use the Softmax classifier to predict the probability of damage category for the high-level feature representation, output the probability of belonging to each damage category, use the cross entropy loss function to calculate the difference between the predicted damage category and the actual damage category, and optimize the model parameters through the back propagation algorithm to obtain a trained damage category recognition model;

[0127] The probability that the output belongs to each damage category is calculated as: Among them, y x = c represents the damage category y of multimodal information xx Belongs to damage category c, p(y x =c|h x ) indicates that given a high-level feature representation h x Under the condition of c , W k They represent the weight vectors of damage category c and damage category k respectively, and Ck is the total number of damage categories;

[0128] The actual damage category is marked in advance by professionals.

[0129] The weight vector of the damage category is the parameter of the Softmax classifier, which is learned and optimized during the training process of the damage category recognition model;

[0130] Step S260: using the trained damage category recognition model to predict the probability that the current multimodal information belongs to each damage category, and taking the damage category with the largest probability value as the damage category of the current multimodal information; the damage categories include crack damage, steel bar corrosion damage, concrete carbonization damage, and fatigue accumulation damage;

[0131] Step S300: Based on the damage category, the pre-trained bridge damage evolution model is used to predict the bridge damage characteristics of adjacent time steps and calculate the damage propagation rate;

[0132] The specific method of predicting the bridge damage characteristics of adjacent time steps based on the damage category and calculating the damage extension rate using the pre-trained bridge damage evolution model is as follows:

[0133] Step S310: measuring bridge damage data corresponding to the multimodal information, and grouping the bridge damage data based on different damage categories output;

[0134] Step S320: for each damage category, extracting its damage feature evolution sequence that changes with time from the bridge damage data; the damage features include crack width, crack length and damage area;

[0135] Step S330: construct a bridge damage evolution model, including an input layer, an LSTM hidden layer and an output layer. The input layer is used to receive the damage feature evolution sequence, input a damage feature at each time step, use the LSTM hidden layer to encode the damage feature of the current time step, and map the output of the LSTM hidden layer to the damage feature prediction value of the next time step;

[0136] The structure of the LSTM hidden layer includes a forget gate, an input gate, an output gate, a candidate memory cell state, a memory cell state and a hidden state. The forget gate is used to control whether the information in the memory cell state of the previous time step is forgotten. The input gate is used to control whether the information in the input of the current time step is added to the memory cell state. The output gate is used to control whether the information in the memory cell state is output to the hidden state. The candidate memory cell state is used to represent the new memory information brought by the input of the current time step. The memory cell state integrates the memory of the damage characteristics of the previous time step and the new memory of the current time step. The hidden state represents the output of the LSTM hidden layer of the current time step, encoding the damage evolution information up to the current time step.

[0137] The update formula of the LSTM hidden layer is: t =σ(W f ×[h t-1 ,x t ]+b f ),i t =σ(W i ×[h t-1 ,x t ]+b i ), o t =σ(W o ×[h t-1 ,x t ]+b o ), h t =o t ×tanh(C t ), where W f , W i , W C , W o are the weight matrices of the forget gate, input gate, candidate memory cell state, and output gate, respectively, and b f , b i , b C , b o are the bias vectors of the forget gate, input gate, candidate memory cell state, and output gate, respectively. σ(·) is the Sigmoid activation function, tanh(·) is the hyperbolic tangent activation function, and h t-1 、h t Represent the hidden states of the previous time step and the current time step, respectively, x t represents the damage characteristics of the current time step, f t 、i t , C t , o t They represent the forget gate, input gate, candidate memory unit state, memory unit state, and output gate respectively;

[0138] Step S340: designing a loss function based on the mean square error and the Paris law regularization term, taking the value of minimizing the loss function as the training goal, and training the bridge damage evolution model;

[0139] The loss function is designed based on the mean square error and the Paris law regularization term, and the value of the minimization loss function is used as the training goal. The specific method for training the bridge damage evolution model is:

[0140] Step S341: Obtain damage feature evolution sequence samples and divide them into subsequences of fixed length Z. For each subsequence, use the damage features of the first Z-1 time steps as input, use the damage features of the last time step as prediction targets, and combine the inputs of all subsequences and their corresponding prediction targets into training sample pairs.

[0141] Step S342: using the mean square error to calculate the difference between the true value of the damage feature and the predicted value of the damage feature, introducing the Paris law regularization term, and obtaining the final loss function Loss based on the weighted sum of the mean square error and the Paris law regularization term. Taking the value of the minimization loss function as the training goal, the model parameters are optimized to obtain a trained bridge damage evolution model;

[0142] The calculation formula of the loss function is: Loss = λ 1 ×L MSE +λ 2 ×L Paris , where L MSE is the mean square error, L Paris is the Paris law regularization term, λ 1 , 2 are the weight coefficients of the mean square error and the Paris law regularization term,

[0143] λ 1 , 2 Satisfy λ 1 +λ 2 =1, and the specific value is set by those skilled in the art according to requirements.

[0144] The calculation formula of the mean square error is: Where N is the number of training sample pairs, is the predicted value of the damage feature, x n is the true value of the damage feature, is the square of the L2 norm;

[0145] The calculation formula of the Paris law regularization term is: Among them, λ is the regularization coefficient, are the crack lengths predicted at the z+1th and zth time steps, ΔN z is the cycle increment of the zth time step, ΔK z is the stress intensity factor range of the zth time step, CL, m represents material constants; Z represents the time step.

[0146] The cycle number increment represents the number of times the fatigue load acts in the zth time step, and is directly obtained through multimodal information according to the strain sensor record;

[0147] The stress intensity factor range represents the variation range of the stress field at the crack tip, which is obtained by those skilled in the art based on finite element analysis;

[0148] The material constants are related to the fatigue properties of the bridge material and are obtained from the material handbook.

[0149] Step S350: Use the trained bridge damage evolution model to predict damage characteristics, obtain the damage characteristic change Δx of adjacent time steps, and calculate the damage extension rate v x ;

[0150] The damage growth rate is calculated as follows: in, are the predicted values ​​of damage characteristics at adjacent time steps t and t+1, respectively, and Δt is the time step;

[0151] Step S400: dynamically divide the local grid of the bridge based on the optimized sensor network, obtain damage location results and calculate the degree of bridge structure degradation, and evaluate the remaining life of the bridge in combination with the damage extension rate;

[0152] The specific method of dynamically dividing the local grid of the bridge based on the optimized sensor network, obtaining the damage location result and calculating the degree of deterioration of the bridge structure, and evaluating the remaining life of the bridge in combination with the damage extension rate is as follows:

[0153] Step S410: obtaining the geometric topological relationship of the bridge structure and obtaining the grid size Δp of the optimized sensor network;

[0154] Step S420: dynamically adjusting the local grid size according to the intensity of the acoustic emission signal in the multimodal information. When the intensity of the acoustic emission signal of the grid unit in the optimized sensor network is greater than the upper limit of the intensity threshold, the local grid size Δp' is adjusted to half of the original grid size; when the intensity of the acoustic emission signal of the grid unit in the optimized sensor network is less than the lower limit of the intensity threshold, the local grid size Δp' is adjusted to twice the original grid size;

[0155] The values ​​of the upper limit of the intensity threshold and the lower limit of the intensity threshold are set by those skilled in the art based on experience.

[0156] Step S430: define the acoustic wave propagation path compensation term Δs, calculate the difference between the recorded acoustic emission signal arrival time and the acoustic wave propagation path compensation term, and obtain the corrected acoustic emission signal arrival time s';

[0157] The calculation formula of the sound wave propagation path compensation term is: Among them, d j and v j are the propagation distance and speed of the acoustic emission signal on the jth propagation path, respectively, and J is the number of propagation path segments;

[0158] The calculation formula of the corrected acoustic emission signal arrival time is: s'=s-Δs, where s is the recorded acoustic emission signal arrival time;

[0159] Step S440: Preliminarily locate the acoustic emission signal on the grid size Δp to obtain a suspected damage area, and obtain a damage location result in a local grid of the suspected damage area;

[0160] The preliminary positioning adopts an acoustic emission positioning algorithm. Preferably, a TDOA algorithm is selected to calculate the distance between the sound source and multiple sensors, and a distance difference equation group is constructed through the arrival time difference of the acoustic emission signals received by the multiple sensors to solve the coordinates of the sound source to obtain a preliminary positioning result;

[0161] Obtaining the damage location result in the local grid of the suspected damage area means selecting the suspected damage area on the basis of preliminary location, locating it on the corresponding local grid, and obtaining the precise location of the damage by obtaining more acoustic emission signal arrival time differences and combining the acoustic wave propagation path compensation item;

[0162] The specific method for preliminarily locating the acoustic emission signal on the grid size Δp to obtain the suspected damage area is:

[0163] Step S441: Assume that the position of the sound source is (xi, yi, zi), the position of the ni-th sensor is (xni, yni, zni), and the sound wave propagation speed is vi;

[0164] Step S442: Calculate the distance dni from the sound source to the ni-th sensor, and calculate the time tni when the acoustic emission signal reaches the ni-th sensor;

[0165] The calculation formula for the time tni when the acoustic emission signal reaches the nith sensor is: Among them, t0 is the emission time of the acoustic emission signal;

[0166] Step S443: for any two sensors ni, nj, calculate the time difference Δtij between them receiving the acoustic emission signals;

[0167] The calculation formula of the time difference is:

[0168] Step S444: Substitute the distance into the calculation formula of the time difference and obtain the time difference equation group, solve the time difference equation group, obtain the sound source position as the preliminary positioning result, and obtain the suspected damage area;

[0169] The time difference equation group is: dn2-dn1=vi×Δt21, dn3-dn1=vi×Δt31, ..., dnN-dn1=vi×ΔtN1, where dnN is the distance from the acoustic emission signal to the nNth sensor, nN is the number of sensors, and ΔtN1 represents the time difference between sensors n1 and nN receiving the acoustic emission signal;

[0170] The method for solving the time difference equations is Newton's iteration method or gradient descent method;

[0171] The specific method for obtaining the damage location result in the local grid of the suspected damage area is:

[0172] Step S445: Assume that M sensors are deployed on the local grid, and the position of the mi-th sensor is (x mi ,y mi ,z mi ), for any two sensors mi and mj, they form a hyperboloid with the sound source, satisfying the hyperboloid equation:

[0173] Step S446: Combined with the sound wave propagation path compensation term Δs, the hyperbolic equation is modified to: dmi-dmj=Δs×vi×Δt ij ;

[0174] Step S447: selecting multiple pairs of sensors, constructing multiple hyperbolic surface equations, forming an equation group, and solving the equation group to obtain the precise location of the sound source as a damage localization result;

[0175] Step S450: Count the number of damage points according to the damage location results and calculate the damage density l af , according to the time delay and energy attenuation of the acoustic emission signal propagating in the bridge structure, the degree of deterioration of the bridge structure is calculated;

[0176] The calculation formula for the degree of deterioration of the bridge structure is: DI = w 1 × af +w 2 ×ΔtI+w 3 ×EI, where ΔtI is the time delay, EI is the energy attenuation, and w 1 、w 2 、w3 are the weight coefficients of damage density, time delay and energy attenuation, respectively, and the sum of the weight coefficients is 1;

[0177] The weight coefficients of the damage density, time delay and energy attenuation are set by those skilled in the art based on experience.

[0178] Step S460: Calculate the remaining life Lr of the bridge structure according to the bridge structure degradation degree and damage extension rate;

[0179] The calculation formula of the remaining life is: Among them, DI max is the critical threshold of the bridge structure deterioration degree, indicating the degree of deterioration when the bridge structure fails completely;

[0180] The critical threshold of the degree of deterioration of the bridge structure is determined by those skilled in the art according to the design and construction of the bridge and based on actual conditions.

[0181] Figure 3 Flowchart of the bridge structure remaining life assessment method provided for this application.

[0182] Step S500: construct a maintenance strategy knowledge base, and generate a maintenance strategy set according to the bridge damage category, remaining life and bridge structure deterioration degree;

[0183] The specific method of constructing the maintenance strategy knowledge base and generating a maintenance strategy set according to the bridge damage category, the remaining life and the degree of bridge structure deterioration is as follows:

[0184] Step S510: collecting expert knowledge in the field of bridge maintenance, including the corresponding relationship between damage categories, bridge structure degradation degree, remaining life and maintenance strategies, and constructing a maintenance strategy knowledge base;

[0185] The rules in the maintenance strategy knowledge base are as follows: IF damage category = X AND bridge structure deterioration degree = YAND remaining life = Z THEN maintenance strategy = S;

[0186] Step S520: Taking the bridge damage category, remaining life and bridge structure deterioration degree as input, matching feasible maintenance strategies in the maintenance strategy knowledge base, generating a maintenance strategy set, and sending it to maintenance personnel.

[0187] Example 2

[0188] like Figure 4 As shown, the multi-modal information fusion steel-UHPC composite bridge dynamic maintenance system provided by this application includes:

[0189] The multimodal information acquisition module obtains the optimized sensor network and monitors the multimodal information based on the sensor layout optimization model, sensor collaborative layout optimization model and sensor quantity dynamic adjustment method;

[0190] The damage category output module, based on the pre-trained damage category recognition model, inputs multimodal information and outputs damage categories;

[0191] The damage extension calculation module uses the pre-trained bridge damage evolution model to predict the bridge damage characteristics of adjacent time steps based on the damage category and calculate the damage extension rate;

[0192] The remaining life assessment module dynamically divides the local grid of the bridge based on the optimized sensor network, obtains the damage location results and calculates the degree of deterioration of the bridge structure, and evaluates the remaining life of the bridge in combination with the damage extension rate;

[0193] The maintenance strategy generation module is used to build a maintenance strategy knowledge base and generate a set of maintenance strategies based on bridge damage categories, remaining life, and bridge structure deterioration degree.

[0194] Example 3

[0195] The present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the processor executes the steps in the dynamic maintenance method of a steel-UHPC composite bridge with multimodal information fusion.

[0196] The present application provides a readable storage medium. Computer-readable instructions are stored on the readable storage medium. When the computer-readable instructions are executed by a processor, a dynamic maintenance method for a steel-UHPC composite bridge with multimodal information fusion according to an embodiment of the present application can be executed. The storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory and cache memory. Non-volatile memory may include, for example, read-only memory, hard disk, flash memory, etc.

[0197] In addition, according to the implementation of the present application, the process described above can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be run by a processor to execute instructions corresponding to the method steps provided by the present application, for example: based on the sensor layout optimization model, the sensor collaborative layout optimization model and the sensor quantity dynamic adjustment method, an optimized sensor network is obtained to monitor multimodal information; based on the pre-trained damage category recognition model, multimodal information is input and damage categories are output; based on the damage category, the pre-trained bridge damage evolution model is used to predict the bridge damage characteristics of adjacent time steps and calculate the damage extension rate; based on the optimized sensor network, the bridge is dynamically divided into local grids, the damage location results are obtained and the degree of bridge structure degradation is calculated, and the remaining life of the bridge is evaluated in combination with the damage extension rate; a maintenance strategy knowledge base is constructed, and a maintenance strategy set is generated according to the bridge damage category, the remaining life and the degree of bridge structure degradation. When the computer program is executed by the central processing unit, the above functions defined in the method of the present application are executed.

[0198] The method and system of the present application can be implemented in a variety of ways. The above order of steps for the method is for illustration only, and the method steps of the present application are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present application can also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers a recording medium storing a program for executing the method according to the present application.

[0199] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0200] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A dynamic maintenance method for steel-UHPC composite bridges based on multi-modal information fusion, characterized in that: include: Based on the sensor deployment optimization model, sensor collaborative deployment optimization model and sensor quantity dynamic adjustment method, an optimized sensor network is obtained to monitor multimodal information; Based on the pre-trained damage classification recognition model, multi-modal information is input and damage classification is output; Based on the damage category, the pre-trained bridge damage evolution model is used to predict the bridge damage characteristics of adjacent time steps and calculate the damage propagation rate; Based on the optimized sensor network, the local grid of the bridge is dynamically divided to obtain the damage location results and calculate the degree of deterioration of the bridge structure, and the remaining life of the bridge is evaluated in combination with the damage extension rate; A maintenance strategy knowledge base is constructed to generate a set of maintenance strategies based on bridge damage categories, remaining life, and degree of bridge structure deterioration.

2. The multi-modal information fusion dynamic maintenance method for steel-UHPC composite bridge according to claim 1 is characterized in that: The specific method for obtaining an optimized sensor network and monitoring multimodal information based on the sensor deployment optimization model, the sensor collaborative deployment optimization model and the sensor quantity dynamic adjustment method is as follows: Define the damage location accuracy requirement matrix; Define a sensor deployment decision matrix to represent the number of sensors deployed in each area; Taking minimizing the sensor deployment cost as the optimization objective function and the number of sensors as the optimization variable, the positioning accuracy contribution constraint of the sensors is defined, and a sensor deployment optimization model is constructed. The initial optimal number of sensors in each area is obtained through the optimization algorithm. Define a damage location result matrix to represent the damage density obtained in each damage location in each area; A method for dynamically adjusting the number of sensors is designed to update the number of sensors according to the relationship between the damage density and the preset damage density threshold upper limit and damage density threshold lower limit; Define a heterogeneous sensor collaborative decision matrix to represent the number of sensor pairs of different types deployed at the same time; Taking maximizing the contribution of sensor collaborative positioning as the optimization objective function and the number of sensor pairs deployed simultaneously as the optimization variable, the sensor pair number constraint is defined, and a sensor collaborative deployment optimization model is established. The optimal number of sensor pairs is obtained through the optimization algorithm. Based on the optimal number of sensors, the method for dynamically adjusting the number of sensors and the optimal number of sensor pairs, an optimized sensor network is obtained, and multimodal information is collected based on the sensor network; the multimodal information includes acoustic emission signals, vibration signals, strain signals, and image data.

3. The multi-modal information fusion dynamic maintenance method for steel-UHPC composite bridge according to claim 2 is characterized in that: The method for dynamically adjusting the number of sensors is as follows: dynamically adjusting the sensor deployment decision matrix according to the damage location result matrix, presetting the upper limit of the damage density threshold and the lower limit of the damage density threshold, and for each area, when the damage density of the sensors in the area is greater than the upper limit of the damage density threshold, updating the number of sensors in the area, and the updated number of sensors is the sum of the current number of sensors in the area and the sensor number adjustment step; When the damage density of sensors in the area is less than the lower limit of the damage density threshold, the difference between the current number of sensors in the area and the sensor number adjustment step is calculated, the difference is compared with the lower limit of the sensor number, and the maximum value between the two is selected as the updated sensor number.

4. The multi-modal information fusion dynamic maintenance method for steel-UHPC composite bridge according to claim 3 is characterized in that: The specific method of the damage category recognition model based on pre-training, inputting multimodal information, and outputting damage categories is: Constructing a damage category recognition model, wherein the damage category recognition model includes a feature extraction module, an attention fusion module, a deep belief network module, and a loss classification module; Preprocess the multimodal information and extract multimodal features based on the feature extraction module; The attention mechanism is used to calculate the weights of multimodal features, and the weighted multimodal features are fused based on the attention fusion module to construct a multimodal feature vector. The multimodal feature vector is input into the deep belief network module to learn the high-level feature representation h x ; The high-level feature representation is input into the loss classification module, and the Softmax classifier is used to predict the probability of damage category of the high-level feature representation. The probability of belonging to each damage category is output, and the difference between the predicted damage category and the actual damage category is calculated using the cross entropy loss function. The model parameters are optimized through the back propagation algorithm to obtain a trained damage category recognition model. The trained damage category recognition model is used to predict the probability that the current multimodal information belongs to each damage category, and the damage category with the largest probability value is taken as the damage category of the current multimodal information; the damage categories include crack damage, steel corrosion damage, concrete carbonization damage, and fatigue accumulation damage.

5. The multi-modal information fusion dynamic maintenance method for steel-UHPC composite bridge according to claim 4, characterized in that: The specific method of predicting the bridge damage characteristics of adjacent time steps based on the damage category and calculating the damage extension rate using the pre-trained bridge damage evolution model is as follows: Measure the bridge damage data corresponding to the multimodal information, and group the bridge damage data based on the output different damage categories; For each damage category, the damage characteristic evolution sequence varying with time is extracted from the bridge damage data; the damage characteristics include crack width, crack length and damage area; A bridge damage evolution model is constructed, including an input layer, an LSTM hidden layer, and an output layer. The input layer is used to receive the damage feature evolution sequence. A damage feature is input at each time step. The LSTM hidden layer is used to encode the damage feature of the current time step, and the output of the LSTM hidden layer is mapped to the damage feature prediction value of the next time step. The loss function is designed based on the mean square error and the Paris law regularization term, and the bridge damage evolution model is trained by minimizing the value of the loss function as the training goal; The trained bridge damage evolution model is used to predict damage characteristics, obtain the damage characteristic change Δx of adjacent time steps, and calculate the damage extension rate v x .

6. The multi-modal information fusion dynamic maintenance method for steel-UHPC composite bridge according to claim 5, characterized in that: The loss function is designed based on the mean square error and the Paris law regularization term, and the value of the minimization loss function is used as the training goal. The specific method for training the bridge damage evolution model is: Get the damage feature evolution sequence samples and divide them into subsequences of fixed length Z. For each subsequence, take the damage features of the first Z-1 time steps as input, and take the damage features of the last time step as the prediction target. Combine the inputs of all subsequences and their corresponding prediction targets into training sample pairs. The mean square error is used to calculate the difference between the true value of the damage feature and the predicted value of the damage feature. The Paris law regularization term is introduced. The final loss function Loss is obtained based on the weighted sum of the mean square error and the Paris law regularization term. The training goal is to minimize the value of the loss function, optimize the model parameters, and obtain a trained bridge damage evolution model.

7. The multi-modal information fusion dynamic maintenance method for steel-UHPC composite bridge according to claim 6, characterized in that: The specific method of dynamically dividing the local grid of the bridge based on the optimized sensor network, obtaining the damage location result and calculating the degree of deterioration of the bridge structure, and evaluating the remaining life of the bridge in combination with the damage extension rate is as follows: Obtain the geometric topological relationship of the bridge structure and obtain the grid size Δp of the optimized sensor network; The local grid size is dynamically adjusted according to the intensity of the acoustic emission signal in the multimodal information. When the intensity of the acoustic emission signal of the grid unit in the optimized sensor network is greater than the upper limit of the intensity threshold, the local grid size Δp' is adjusted to half of the original grid size; when the intensity of the acoustic emission signal of the grid unit in the optimized sensor network is less than the lower limit of the intensity threshold, the local grid size Δp' is adjusted to twice the original grid size; Define the acoustic wave propagation path compensation term Δs, calculate the difference between the recorded acoustic emission signal arrival time and the acoustic wave propagation path compensation term, and obtain the corrected acoustic emission signal arrival time s'; The acoustic emission signal is preliminarily located on the grid size Δp to obtain the suspected damage area, and the damage location result is obtained in the local grid of the suspected damage area; According to the damage location results, the number of damage points is counted and the damage density l is calculated. af , according to the time delay and energy attenuation of the acoustic emission signal propagating in the bridge structure, the degree of deterioration of the bridge structure is calculated; The remaining life Lr of the bridge structure is calculated based on the deterioration degree and damage extension rate of the bridge structure.

8. The multi-modal information fusion dynamic maintenance method for steel-UHPC composite bridge according to claim 7, characterized in that: The specific method for preliminarily locating the acoustic emission signal on the grid size Δp to obtain the suspected damage area is: Assume that the position of the sound source is (xi,yi,zi), the position of the ni-th sensor is (xni,yni,zni), and the speed of sound wave propagation is vi; Calculate the distance dni from the sound source to the ni-th sensor, and calculate the time tni when the acoustic emission signal reaches the ni-th sensor; For any two sensors ni, nj, calculate the time difference Δtij between them receiving the acoustic emission signal; Substitute the distance into the time difference calculation formula and obtain the time difference equation group. Solve the time difference equation group to obtain the sound source position as the preliminary positioning result and obtain the suspected damage area.

9. A dynamic maintenance system for steel-UHPC composite bridges with multimodal information fusion, which is used to implement the dynamic maintenance method for steel-UHPC composite bridges with multimodal information fusion as claimed in any one of claims 1 to 8, characterized in that: include: The multimodal information acquisition module obtains the optimized sensor network and monitors the multimodal information based on the sensor layout optimization model, sensor collaborative layout optimization model and sensor quantity dynamic adjustment method; The damage category output module, based on the pre-trained damage category recognition model, inputs multimodal information and outputs damage categories; The damage extension calculation module uses the pre-trained bridge damage evolution model to predict the bridge damage characteristics of adjacent time steps based on the damage category and calculate the damage extension rate; The remaining life assessment module dynamically divides the local grid of the bridge based on the optimized sensor network, obtains the damage location results and calculates the degree of deterioration of the bridge structure, and evaluates the remaining life of the bridge in combination with the damage extension rate; The maintenance strategy generation module is used to build a maintenance strategy knowledge base and generate a set of maintenance strategies based on bridge damage categories, remaining life, and bridge structure deterioration degree.

10. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the dynamic maintenance method of a steel-UHPC composite bridge with multimodal information fusion as described in any one of claims 1 to 8 are implemented.

Images