Steel-UHPC composite bridge dynamic maintenance system and method based on multi-modal information fusion

Abstract

The application relates to a steel-UHPC combined bridge dynamic maintenance system and method based on multi-modal information fusion, and relates to the technical field of crack detection. Based on a sensor layout optimization model, a sensor collaborative layout optimization model and a sensor quantity dynamic adjustment method, an optimized sensor network is obtained to monitor multi-modal information. Based on a pre-trained damage category identification model, multi-modal information is input, and a damage category is output. Based on the damage category, a pre-trained bridge damage evolution model is used to predict the bridge damage characteristics at adjacent time steps, and a damage propagation rate is calculated. Based on the optimized sensor network, a local grid of the bridge is dynamically divided, a damage positioning result is obtained, and the degradation degree of the bridge structure is calculated, the remaining life of the bridge is evaluated in combination with the damage propagation rate, a maintenance strategy knowledge base is constructed, a maintenance strategy set is generated according to the bridge damage category, the remaining life and the degradation degree of the bridge structure, and the precision of bridge damage detection and maintenance prevention is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of crack detection, and in particular to a steel-UHPC combined bridge dynamic maintenance system and method based on multi-modal information fusion. BACKGROUND

[0002] Steel-UHPC combined bridges combine the high strength of steel structures and the excellent durability of ultra-high performance concrete. The two materials play a synergistic role through precise interface connection, and have high bearing capacity, fatigue resistance and long service life. However, during long-term service, steel-UHPC combined bridges may be affected by multiple factors such as vehicle load, fatigue damage, environmental erosion and material aging, leading to performance degradation and threatening structural safety.

[0003] The patent application with publication number CN117952601A discloses a bridge health big data intelligent management and maintenance system, which includes a detection module including a detector for collecting bridge data and environmental data; the bridge data includes bridge structure data and bridge feature data; the bridge structure data includes spatial information of the bridge, and the bridge feature data includes surface information of the bridge; a mobile module and a fixed module are included in a mobile module; wherein the detection device is arranged on the back side of the bridge through the fixed module, and the detection device is movably arranged on the back side of the bridge through the mobile module and the fixed module, and the detection device is used for detecting cracks on the back side of the bridge.

[0004] Existing methods rely on single crack data for analysis and fail to combine multi-modal information, making it difficult to accurately capture damage to the multi-material system of the bridge. SUMMARY

[0005] The present application aims to at least partially solve one of the technical problems in the related art. To this end, one object of the present application is to provide a steel-UHPC combined bridge dynamic maintenance system and method based on multi-modal information fusion, which improves the accuracy of bridge damage detection and maintenance prevention.

[0006] One aspect of the present application provides a steel-UHPC combined bridge dynamic maintenance method based on multi-modal information fusion, comprising:

[0007] Step S100: based on a sensor layout optimization model, a sensor collaborative layout optimization model and a sensor number dynamic adjustment method, an optimized sensor network is obtained to monitor multi-modal information;

[0008] Step S200: based on a pre-trained damage category recognition model, input the multi-modal information, and output the damage category;

[0009] Step S300: Based on the damage category, the pre-trained bridge damage evolution model is used to predict the bridge damage features at the adjacent time step, and the damage propagation rate is calculated;

[0010] Step S400: Based on the optimized sensor network, the bridge is dynamically divided into local grids, the damage positioning result is obtained, and the bridge structure degradation degree is calculated, and the bridge remaining life is evaluated in combination with the damage propagation rate;

[0011] Step S500: Construct a maintenance strategy knowledge base, and generate a maintenance strategy set according to the bridge damage category, the remaining life and the bridge structure degradation degree;

[0012] The sensor layout optimization model, the sensor collaborative layout optimization model and the sensor number dynamic adjustment method obtain an optimized sensor network, and the specific method for monitoring multi-modal information is:

[0013] Step S110: Define a damage positioning accuracy requirement matrix;

[0014] Step S120: Define a sensor layout decision matrix for representing the number of sensors laid in each region;

[0015] Step S130: Taking minimizing the sensor layout cost as an optimization objective function, taking the number of sensors as an optimization variable, defining a positioning accuracy contribution constraint of the sensor, constructing a sensor layout optimization model, and solving the initial optimal number of sensors in each region through an optimization algorithm;

[0016] Step S140: Define a damage positioning result matrix for representing the damage intensity obtained by each damage positioning in each region;

[0017] Step S150: Design a sensor number dynamic adjustment method, and update the number of sensors according to the relationship between the damage intensity and the preset upper and lower damage intensity thresholds;

[0018] Step S160: Define a heterogeneous sensor collaborative decision matrix for representing the number of sensor pairs laid at the same time;

[0019] Step S170: Taking maximizing the sensor collaborative positioning contribution as an optimization objective function, taking the number of sensor pairs laid at the same time as an optimization variable, defining a sensor pair number constraint, establishing a sensor collaborative layout optimization model, and solving the optimal number of sensor pairs through an optimization algorithm;

[0020] Step S180: Based on the optimal number of sensors, the sensor number dynamic adjustment method and the optimal number of sensor pairs, an optimized sensor network is obtained, and multi-modal information is collected based on the sensor network; the multi-modal information includes acoustic emission signals, vibration signals, strain signals and image data;

[0021] The sensor quantity dynamic adjustment method is to dynamically adjust the sensor layout decision matrix according to the damage positioning result matrix, preset an upper limit of damage density threshold and a lower limit of damage density threshold, for each region, when the damage density of the sensor in the region is greater than the upper limit of the damage density threshold, the sensor quantity of the region is updated, and the updated sensor quantity is the sum of the current sensor quantity of the region and the sensor quantity adjustment step; when the damage density of the sensor in the region is less than the lower limit of the damage density threshold, the difference between the current sensor quantity of the region and the sensor quantity adjustment step is calculated, the difference and the lower limit of the sensor quantity are compared, and the maximum value between the two is selected as the updated sensor quantity;

[0022] The specific method for inputting multi-modal information and outputting damage category based on the pre-trained damage category recognition model is:

[0023] Step S210: constructing a damage category recognition model, the damage category recognition model comprising a feature extraction module, an attention fusion module, a deep belief network module, and a loss classification module;

[0024] Step S220: pre-processing the multi-modal information, and extracting multi-modal features based on the feature extraction module;

[0025] Step S230: calculating the weight of the multi-modal features using an attention mechanism, fusing the weighted multi-modal features based on the attention fusion module, and constructing a multi-modal feature vector;

[0026] Step S240: inputting the multi-modal feature vector into the deep belief network module to learn a high-level feature representation h x ;

[0027] Step S250: inputting the high-level feature representation into the loss classification module, using a Softmax classifier to predict the probability of the high-level feature representation belonging to each damage category, outputting the probability of belonging to each damage category, calculating the difference between the predicted damage category and the actual damage category using a cross-entropy loss function, and optimizing the model parameters through a back propagation algorithm to obtain a trained damage category recognition model;

[0028] Step S260: predicting the probability of the current multi-modal information belonging to each damage category using the trained damage category recognition model, and taking the damage category with the maximum probability value as the damage category of the current multi-modal information; the damage categories include crack damage, reinforcement corrosion damage, concrete carbonation damage, and fatigue accumulation damage;

[0029] The specific method for predicting the bridge damage feature of the adjacent time step based on the damage category and using the pre-trained bridge damage evolution model is as follows:

[0030] Step S310: measuring the bridge damage data corresponding to the multi-modal information, and grouping the bridge damage data based on the output different damage categories;

[0031] Step S320: for each damage category, extracting the damage feature evolution sequence changing over time from the bridge damage data; the damage feature includes crack width, crack length and damage area;

[0032] Step S330: constructing a bridge damage evolution model including an input layer, an LSTM hidden layer and an output layer, the input layer is used to receive the damage feature evolution sequence, one damage feature is input at each time step, the LSTM hidden layer is used to encode the damage feature of the current time step, and the output of the LSTM hidden layer is mapped to the predicted value of the damage feature of the next time step;

[0033] Step S340: designing a loss function based on mean square error and Paris law regularization term, taking minimizing the value of the loss function as the training target, and training the bridge damage evolution model;

[0034] Step S350: using the trained bridge damage evolution model to predict the damage feature, obtaining the damage feature change Δx of the adjacent time step, and calculating the damage propagation rate v x ;

[0035] The specific method for training the bridge damage evolution model based on the mean square error and the Paris law regularization term to design the loss function is as follows:

[0036] Step S341: obtaining the damage feature evolution sequence sample and dividing it into sub-sequences with a fixed length Z, for each sub-sequence, taking the damage features of the first Z-1 time steps as input, taking the damage feature of the last time step as the prediction target, and combining the input of all sub-sequences and the corresponding prediction target as a training sample pair;

[0037] Step S342: using mean square error to calculate the difference between the true value of the damage feature and the predicted value of the damage feature, introducing a Paris law regularization term, obtaining a final loss function Loss based on the weighted sum of the mean square error and the Paris law regularization term, taking minimizing the value of the loss function as the training target, optimizing the model parameters, and obtaining the trained bridge damage evolution model;

[0038] The specific method for evaluating the remaining life of the bridge structure based on the optimized sensor network is as follows:

[0039] Step S410: Obtain the geometric topological relationship of the bridge structure, and obtain the grid size Δp of the optimized sensor network;

[0040] Step S420: Dynamically adjust the local grid size according to the intensity of the acoustic emission signal in the multi-modal information; when the intensity of the acoustic emission signal of the grid unit in the optimized sensor network is greater than the upper limit of the intensity threshold, the local grid size Δp' is adjusted to half of the original grid size; when the intensity of the acoustic emission signal of the grid unit in the optimized sensor network is less than the lower limit of the intensity threshold, the local grid size Δp' is adjusted to twice the original grid size;

[0041] Step S430: Define the sound wave propagation path compensation term Δs, calculate the difference between the recorded acoustic emission signal arrival time and the sound wave propagation path compensation term, and obtain the corrected acoustic emission signal arrival time s';

[0042] Step S440: Preliminarily locate the acoustic emission signal on the grid size Δp to obtain a suspicious damage area, and obtain a damage positioning result in the local grid of the suspicious damage area;

[0043] Step S450: According to the damage positioning result, the number of damage points is counted, and the damage density l is calculated af According to the time delay and energy attenuation of the acoustic emission signal in the bridge structure, the degradation degree of the bridge structure is calculated;

[0044] Step S460: According to the degradation degree of the bridge structure and the damage propagation rate, the remaining life Lr of the bridge structure is calculated;

[0045] The specific method for preliminarily locating the acoustic emission signal on the grid size Δp to obtain a suspicious damage area is as follows:

[0046] Step S441: Assume that the acoustic source position is (xi, yi, zi), the position of the nth sensor is (xni, yni, zni), and the sound wave propagation speed is vi;

[0047] Step S442: Calculate the distance dni from the acoustic source to the nth sensor, and calculate the time tni at which the acoustic emission signal arrives at the nth sensor;

[0048] Step S443: For any two sensors ni and nj, calculate the time difference Δtij at which they receive the acoustic emission signal;

[0049] Step S444: Substitute the distances into the calculation formula of the time difference and solve the time difference equation set to obtain the sound source position as the preliminary positioning result, and obtain the suspicious damage area;

[0050] The specific method for constructing the maintenance strategy knowledge base according to the bridge damage category, the remaining life and the bridge structure deterioration degree to generate the maintenance strategy set is:

[0051] Step S510: Collect expert knowledge in the field of bridge maintenance, including the correspondence between the damage category, the bridge structure deterioration degree, the remaining life and the maintenance strategy, and construct the maintenance strategy knowledge base;

[0052] Step S520: Take the bridge damage category, the remaining life and the bridge structure deterioration degree as input, match the feasible maintenance strategy in the maintenance strategy knowledge base, generate the maintenance strategy set, and send it to the maintenance personnel.

[0053] One aspect of the present application provides a multi-modal information fusion steel-UHPC composite bridge dynamic maintenance system, comprising:

[0054] A multi-modal information acquisition module obtains an optimized sensor network based on a sensor layout optimization model, a sensor collaborative layout optimization model and a sensor number dynamic adjustment method, and monitors multi-modal information;

[0055] A damage category output module inputs multi-modal information based on a pre-trained damage category recognition model and outputs a damage category;

[0056] A damage expansion calculation module predicts the bridge damage features at adjacent time steps based on the damage category using a pre-trained bridge damage evolution model, and calculates the damage expansion rate;

[0057] A remaining life evaluation module performs local grid dynamic division on the bridge based on the optimized sensor network, obtains damage positioning results and calculates the bridge structure deterioration degree, and evaluates the remaining life of the bridge in combination with the damage expansion rate;

[0058] A maintenance strategy generation module is used to construct a maintenance strategy knowledge base and generate a maintenance strategy set according to the bridge damage category, the remaining life and the bridge structure deterioration degree.

[0059] One aspect of the present application provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to perform the steps in the multi-modal information fusion steel-UHPC composite bridge dynamic maintenance method.

[0060] The multi-modal information fusion steel-UHPC composite bridge dynamic maintenance system and method provided by the present application have the following advantages over the prior art:

[0061] The application comprehensively considers damage positioning accuracy requirements, sensor layout costs and other factors, constructs a sensor layout optimization model, realizes quantitative optimization of sensor layout, introduces a sensor collaborative layout optimization model, considers the collaborative effect of heterogeneous sensors, further improves the monitoring performance of the sensor network, and adaptively adjusts the number of sensors according to real-time damage monitoring results, to realize dynamic optimization of the sensor network.

[0062] The application constructs a damage category recognition model containing feature extraction, attention fusion, deep belief network and other modules, can effectively process and fuse multi-modal monitoring data, adaptively gives different modal features and weights by using an attention mechanism, highlights the role of key modal, and improves the accuracy of damage recognition.

[0063] The application constructs a special damage evolution model for different damage categories, fully considers the evolution characteristics and mechanism of different damage modes, models the damage evolution process by using an LSTM network, can capture the long-term dependence relationship of damage development, improves the accuracy of evolution prediction, introduces a Paris law regularization term in the loss function, integrates the physical mechanism of damage propagation, and improves the physical rationality of damage evolution prediction.

[0064] The application dynamically adjusts the local grid size according to the acoustic emission signal intensity, improves the accuracy of damage positioning while ensuring the calculation efficiency, comprehensively considers the bridge structure degradation degree and damage propagation rate, gives the overall evaluation of the bridge residual life, realizes precise positioning and quantitative evaluation of bridge damage on multiple scales and multiple levels, and predicts the bridge residual life by comprehensively considering the damage degree and evolution trend, to provide comprehensive technical means for performance degradation analysis of the bridge multi-material system.

[0065] The application considers the damage characteristics, degradation law and life expectancy of the bridge multi-material system, gives a comprehensive and systematic maintenance strategy, effectively improves the scientificity and effectiveness of bridge maintenance, and effectively prolongs the service life of the bridge.

[0066] The application is based on multi-modal monitoring data, integrates advanced data analysis, damage identification, life prediction and other methods, and is supplemented by expert knowledge, to form a complete health monitoring and life management solution for the bridge multi-material system. Compared with the existing single crack data dependent method, the application can comprehensively and accurately analyze the bridge damage, master its evolution law, evaluate the structure performance, and give targeted maintenance measures, has significant technical advantages and application prospects in improving the safety of the bridge and prolonging the service life of the bridge. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 A method flowchart of the multi-modal information fusion steel-UHPC composite bridge dynamic maintenance method provided by the application;

[0068] Figure 2 An optimization method flowchart for optimizing a sensor network is provided for the present application;

[0069] Figure 3 A bridge structure residual life evaluation method flowchart is provided for the present application;

[0070] Figure 4 A functional module diagram of a multi-modal information fusion steel-UHPC combined bridge dynamic maintenance system is provided for the present application. DETAILED DESCRIPTION

[0071] For a better understanding of the present application, various aspects of the present application will be described in more detail below with reference to the accompanying drawings. It is to be noted that the detailed description is only a description of exemplary embodiments of the present application and does not limit the scope of the present application in any way. Throughout the specification, like reference numerals refer to like elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0072] In the drawings, the size, dimensions, and shapes of the elements have been slightly adjusted for ease of illustration. The drawings are merely schematic and are not drawn to scale. As used in this document, the terms "substantially", "approximately", and similar terms are used as terms of approximation and not as terms of degree, and are intended to account for the inherent deviations in a measuring or computing process. In addition, in the present application, the order of the steps of the process described does not necessarily indicate the order in which the processes occur in actual operation, unless there is an explicit other limitation or it can be derived from the context.

[0073] It should also be understood that expressions such as "include", "including", "have", "has", "contain" and / or "containing" and the like, are open-ended expressions that are used to specify the presence of stated features, elements, and / or components, but do not preclude the presence or addition of one or more other features, elements, components, and / or combinations thereof. In addition, when expressions such as "at least one of" appear after a list of two or more items, it is meant that any of the listed items can be present, individually or in combination with one or more of the other listed items. Furthermore, when describing embodiments of the present application, the use of "may" means "one or more embodiments of the present application". Also, the use of the term "exemplary" is intended to refer to an example or illustration.

[0074] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0075] It should be noted that the embodiments and the features in the embodiments in the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0076] Embodiment 1

[0077] As shown in the multi-modal information fusion steel-UHPC composite bridge dynamic maintenance method provided by the present application, the method comprises the following steps: Figure 1

[0078] Step S100: obtaining an optimized sensor network based on a sensor layout optimization model, a sensor collaborative layout optimization model and a sensor quantity dynamic adjustment method, and monitoring multi-modal information;

[0079] The specific method for obtaining the optimized sensor network based on the sensor layout optimization model, the sensor collaborative layout optimization model and the sensor quantity dynamic adjustment method, and monitoring multi-modal information is as follows:

[0080] Step S110: defining a damage positioning accuracy requirement matrix;

[0081] The damage positioning accuracy requirement matrix C = [c ab ];wherein c ab represents the positioning accuracy requirement of the a th region for the b th sensor;

[0082] Step S120: defining a sensor layout decision matrix for representing the number of sensors laid in each region;

[0083] The sensor layout decision matrix D = [d ab ];wherein d ab represents the number of the b th sensor laid in the a th region;

[0084] Step S130: defining a positioning accuracy contribution constraint of the sensor, taking the minimization of the sensor layout cost as an optimization objective function, and taking the number of sensors as an optimization variable, constructing a sensor layout optimization model, and solving the initial optimal number of sensors in each region by an optimization algorithm;

[0085] The calculation formula for taking the minimization of the sensor layout cost as the optimization objective function is as follows: wherein cb​ab Cost of deploying the bth kind of sensor for the ath region, Na is the number of regions divided by the bridge, Nb is the number of sensor types;

[0086] The calculation formula of the positioning accuracy contribution constraint of the sensor is: Wherein, f ab represents the positioning accuracy contribution of the bth kind of sensor in the ath region;

[0087] The calculation formula of the positioning accuracy contribution is: Wherein, N total represents the total number of sensors in the ath region, represents the distance between the ith b sensor and the ith g sensor in the ath region;

[0088] Step S140: define a damage positioning result matrix for representing the damage density obtained by each damage positioning in each region;

[0089] The damage positioning result matrix L = [l af ]; wherein, l af represents the damage density of the ath region in the fth positioning;

[0090] The calculation formula of the damage density is: Wherein, n a is the number of damage points detected in the ath region, S a is the area of the ath region;

[0091] The calculation method of the number of damage points is: according to the sensor to obtain the acoustic emission signal, position the damage point, and count the number of damage points;

[0092] Step S150: design a sensor number dynamic adjustment method, update the number of sensors according to the relationship between the damage density and the preset upper limit of the damage density threshold and the lower limit of the damage density threshold;

[0093] The sensor quantity dynamic adjustment method is: dynamically adjusting the sensor layout decision matrix according to the damage positioning result matrix, presetting an upper damage density threshold and a lower damage density threshold, for each region, when the damage density of the sensor in the region is greater than the upper damage density threshold, updating the sensor quantity of the region, and the updated sensor quantity is the sum of the current sensor quantity of the region and a sensor quantity adjustment step; when the damage density of the sensor in the region is less than the lower damage density threshold, calculating the difference between the current sensor quantity of the region and the sensor quantity adjustment step, comparing the difference with a lower sensor quantity limit, and selecting the maximum value between the two as the updated sensor quantity.

[0094] The updated sensor quantity d ab,new is calculated according to the following formula: Wherein, and are the upper damage density threshold and the lower damage density threshold respectively, Δd is the sensor quantity adjustment step, d min is the lower sensor quantity limit, and d ab,old is the sensor quantity before updating.

[0095] The sensor layout optimization model determines the starting point of the sensor layout to meet the initial damage positioning accuracy requirement and the cost constraint; step S150 dynamically adjusts the sensor layout according to the actual damage positioning result in the monitoring process to adapt to the change of the bridge damage state, which realizes the self-adaptability of the sensor network, can increase the sensor quantity in the damage dense area and reduce the sensor quantity in the damage sparse area, so as to meet the positioning accuracy requirement while reducing the cost.

[0096] Step S160: defining a heterogeneous sensor collaborative decision matrix for representing the sensor pair quantity of simultaneously laying different kinds of sensors;

[0097] The heterogeneous sensor collaborative decision matrix Y = [y abg ]; wherein y abg represents the sensor pair quantity of simultaneously laying the bth and gth sensors in the a th region.

[0098] Step S170: taking the maximum sensor collaborative positioning contribution as the optimization objective function, taking the sensor pair quantity of simultaneous laying as the optimization variable, defining the sensor pair quantity constraint, establishing the sensor collaborative layout optimization model, and solving the optimal sensor pair quantity through an optimization algorithm;

[0099] The calculation formula of taking the maximum sensor collaborative positioning contribution as the optimization objective function is: Wherein, eabg represents the cooperative positioning accuracy contribution of the a-th region sensor pair (b, g), (b, g) represents a sensor pair composed of the b-th and g-th sensors;

[0100] The calculation formula of the cooperative positioning accuracy contribution is: wherein, j (b,g) represents the distance of the sensor pair (b, g), Δt (b,g) represents the time difference of the same damage point monitored by the sensor pair (b, g);

[0101] The calculation formula of the sensor pair number constraint is:

[0102] Step S180: Based on the optimal sensor number, the sensor number dynamic adjustment method and the optimal sensor pair number, an optimized sensor network is obtained, and multi-modal information is collected based on the optimized sensor network; the multi-modal information includes acoustic emission signals, vibration signals, strain signals and image data;

[0103] Each grid unit in the optimized sensor network represents a possible sensor layout position, a grid matrix is used to represent the sensor network, each element of the grid matrix corresponds to a grid unit, and the value of the element represents whether a sensor is laid at the position, 1 representing laying and 0 representing not laying;

[0104] Figure 2 An optimization method flowchart of the optimized sensor network provided in the present application is provided;

[0105] The above steps can comprehensively monitor the state of the multi-material system of the bridge through the optimized multi-modal sensor network, overcome the limitations of single damage data, and lay a foundation for precise damage analysis.

[0106] Step S200: Based on the pre-trained damage category recognition model, input the multi-modal information, and output the damage category;

[0107] The specific method of the pre-trained damage category recognition model, inputting the multi-modal information and outputting the damage category, is:

[0108] Step S210: Constructing a damage category recognition model, the damage category recognition model includes a feature extraction module, an attention fusion module, a deep belief network module and a loss classification module;

[0109] The feature extraction module extracts features from multi-source data such as acoustic emission, strain, vibration and image, and obtains feature vectors of different modalities; the attention fusion module uses an attention mechanism to adaptively learn the weights of different modalities, and fuses the weighted features; the deep belief network module uses a pre-trained deep belief network to learn high-level features from the fused features; and the loss classification module uses a Softmax classifier to predict the probability of damage class for the high-level features.

[0110] Step S220: Preprocessing the multi-modal information, extracting multi-modal features based on the feature extraction module;

[0111] The specific method for preprocessing the multi-modal information and extracting the multi-modal features is as follows:

[0112] The acoustic emission signal is subjected to short-time Fourier transform to extract the time-frequency domain features of the acoustic emission signal, including the frequency center and the frequency peak, and the time domain features of the acoustic emission signal, including the energy and the rise time;

[0113] The vibration signal is subjected to wavelet transform to extract the wavelet coefficients as the time-frequency domain features of the vibration signal, and the time domain features of the vibration signal, including the mean value and the peak factor;

[0114] The strain signal is subjected to statistical feature extraction, including the mean value and the variance of the strain signal, and the frequency domain features of the strain signal, including the frequency amplitude and the frequency phase after Fourier transform;

[0115] The image data is subjected to feature extraction to obtain the image semantic features;

[0116] The multi-modal features are composed of the time-frequency domain features of the acoustic emission signal, the time domain features of the acoustic emission signal, the time-frequency domain features of the vibration signal, the time domain features of the vibration signal, the statistical features, the frequency domain features of the strain signal and the image semantic features;

[0117] Step S230: Using an attention mechanism to calculate the weights of the multi-modal features, and fusing the weighted multi-modal features based on the attention fusion module to construct a multi-modal feature vector;

[0118] The calculation formula of the weights of the multi-modal features is as follows: Wherein, f(·) is an attention scoring function, wherein, α i represents the weight of the i-th multi-modal feature, x i , x i' are the i-th and i'-th multi-modal features respectively, and M is the number of modalities of the multi-modal features;

[0119] The multi-modal feature vector is a high-dimensional feature vector, and features of different modalities are weighted and summed, the weight is adaptively learned by an attention mechanism of a damage class recognition model, features of different modalities are mapped to a common feature space to obtain a fused multi-modal feature vector.

[0120] The attention scoring function is a multi-layer perceptron, which is used to calculate the importance score of the multi-modal feature, the multi-modal feature is input into the multi-layer perceptron, and a feature representation is output, the feature representation is mapped to an importance score through a fully connected layer, and the importance score of the multi-modal feature is normalized through a softmax function to obtain an attention weight.

[0121] The process of inputting the multi-modal feature into the multi-layer perceptron and outputting the feature representation is:

[0122] wherein, is a feature representation output by an Lth hidden layer of the multi-layer perceptron, W L , and b L is a weight matrix and a bias vector of the Lth hidden layer, L is a number of fully connected layers of the multi-layer perceptron, and sigma(·) is an activation function;

[0123] The mapping of the feature representation through the fully connected layer to an importance score representation is: wherein, el i is an importance score of an ith multi-modal feature, v T represents a transpose of a weight vector of the fully connected layer, and b represents a bias term.

[0124] The weight matrix and the bias vector of the hidden layer of the multi-layer perceptron, the weight vector of the fully connected layer, and the bias term are model parameters, which are learned and optimized in a training process.

[0125] Step S240: inputting the multi-modal feature vector into a deep belief network module to learn a high-level feature representation h x .

[0126] Step S250: inputting the high-level feature representation into a loss classification module, using a Softmax classifier to perform probability prediction of a damage class on the high-level feature representation, outputting a probability belonging to each damage class, using a cross-entropy loss function to calculate a difference between a predicted damage class and an actual damage class, and optimizing model parameters through a back propagation algorithm to obtain a trained damage class recognition model.

[0127] The calculation formula of the probability belonging to each damage class is: wherein, y x =c represents a damage class y of the multi-modal information xx P(y x =c|h x ) represents the probability that the multi-modal information belongs to the damage class c given the high-level feature representation h x , W c , W k represent the weight vectors of the damage class c, the damage class k respectively, and Ck is the total number of damage classes;

[0128] The real damage class is labeled in advance by a professional.

[0129] The weight vector of the damage class is the parameter of the Softmax classifier, which is learned and optimized during the training process of the damage class recognition model;

[0130] Step S260: using the trained damage class recognition model to predict the probability that the current multi-modal information belongs to each damage class, and taking the damage class with the largest probability value as the damage class of the current multi-modal information; the damage class includes crack class damage, steel bar corrosion class damage, concrete carbonation class damage, and fatigue accumulation class damage;

[0131] Step S300: based on the damage class, using the pre-trained bridge damage evolution model to predict the bridge damage feature at the adjacent time step, and calculating the damage propagation rate;

[0132] The specific method for predicting the bridge damage feature at the adjacent time step based on the damage class and using the pre-trained bridge damage evolution model to calculate the damage propagation rate is as follows:

[0133] Step S310: measuring the bridge damage data corresponding to the multi-modal information, and grouping the bridge damage data based on the output different damage classes;

[0134] Step S320: for each damage class, extracting the damage feature evolution sequence thereof varying with time from the bridge damage data; the damage feature includes crack width, crack length, and damage area;

[0135] Step S330: constructing a bridge damage evolution model, including an input layer, an LSTM hidden layer, and an output layer, the input layer being used to receive the damage feature evolution sequence, one damage feature being input at each time step, the LSTM hidden layer being used to encode the damage feature at the current time step, and the output of the LSTM hidden layer being mapped to the damage feature prediction value at the next time step;

[0136] The structure of the LSTM hidden layer includes a forget gate, an input gate, an output gate, candidate memory cell states, memory cell states, and a hidden state. The forget gate controls whether the information in the memory cell state of the previous time step is forgotten. The input gate controls whether the information in the input of the current time step is added to the memory cell state. The output gate controls whether the information in the memory cell state is output to the hidden state. The candidate memory cell state represents the new memory information brought about by the input of the current time step. The memory cell state integrates the memory of the damage features of the previous time step and the new memory of the current time step. The hidden state represents the output of the LSTM hidden layer at the current time step and encodes the damage evolution information up to the current time step.

[0137] The update formula for the LSTM hidden layer is: f t =σ(W f ×[h t-1 ,x t ]+b f ), i t =σ(W i ×[h t-1 ,x t ]+b i ), o t =σ(W o ×[h t-1 ,x t ]+b o ), h t =o t ×tanh(C t ), where W f W i W C W o b represents the weight matrices for the forget gate, input gate, candidate memory cell states, and output gate, respectively. f b i b C b o Let be the bias vectors for the forget gate, input gate, candidate memory cell state, and output gate, respectively; σ(·) be the sigmoid activation function; tanh(·) be the hyperbolic tangent activation function; and h be the bias vectors for the output gate, input gate, candidate memory cell state, and output gate, respectively. t-1 h t Let x represent the hidden states of the previous time step and the current time step, respectively. t f represents the damage characteristics at the current time step. t i t , C t o t These represent the forget gate, input gate, candidate memory unit state, memory unit state, and output gate, respectively.

[0138] Step S340: Design a loss function based on the mean square error and the Paris law regularization term, and train the bridge damage evolution model by minimizing the value of the loss function as the training target;

[0139] The specific method for training the bridge damage evolution model by designing a loss function based on the mean square error and the Paris law regularization term and minimizing the value of the loss function as the training target is:

[0140] Step S341: Obtain a damage feature evolution sequence sample and divide it into sub-sequences of a fixed length Z, for each sub-sequence, take the damage features of the previous Z-1 time steps as input, take the damage features of the last time step as the prediction target, and combine the input and the corresponding prediction target of all sub-sequences into a training sample pair;

[0141] Step S342: Calculate the difference between the true value of the damage feature and the predicted value of the damage feature using the mean square error, introduce the Paris law regularization term, and obtain the final loss function Loss based on the weighted sum of the mean square error and the Paris law regularization term, to minimize the value of the loss function as the training target, optimize the model parameters, and obtain the trained bridge damage evolution model;

[0142] The calculation formula of the loss function is: Loss = λ1 x L MSE + λ2 x L Paris , wherein L MSE is the mean square error, L Paris is the Paris law regularization term, λ1 and λ2 are weight coefficients of the mean square error and the Paris law regularization term, respectively,

[0143] λ1 and λ2 satisfy λ1 + λ2 = 1, and the specific values are set by a person skilled in the art according to requirements.

[0144] The calculation formula of the mean square error is: , wherein N is the number of training sample pairs, is the predicted value of the damage feature, x n is the true value of the damage feature, is the square of the L2 norm;

[0145] The calculation formula of the Paris law regularization term is: , wherein λ is a regularization coefficient, are the predicted crack lengths of the z+1th and zth time steps, ΔN z is the cycle increment of the zth time step, and ΔK zLet Z represent the range of stress intensity factors at the z-th time step, where CL and m represent material constants, and Z represents the time step.

[0146] The increment of the number of cycles represents the number of times the fatigue load is applied within the z-th time step, which is obtained directly from multimodal information based on the records of the strain sensor.

[0147] The stress intensity factor range represents the range of stress field variation at the crack tip, which is obtained by those skilled in the art based on finite element analysis.

[0148] The material constants mentioned are related to the fatigue performance of bridge materials and are obtained from material handbooks.

[0149] Step S350: Predict damage characteristics using the trained bridge damage evolution model, obtain the damage characteristic changes Δx between adjacent time steps, and calculate the damage propagation rate v. x ;

[0150] The formula for calculating the damage propagation rate is: in, These are the predicted values ​​of the damage features at adjacent time steps t and t+1, respectively, where Δt is the time step size;

[0151] Step S400: Based on the optimized sensor network, the bridge is dynamically divided into local meshes, the damage location results are obtained, the degree of bridge structural deterioration is calculated, and the remaining life of the bridge is evaluated in combination with the damage propagation rate.

[0152] The specific method for dynamically dividing the bridge into local meshes based on an optimized sensor network, obtaining damage location results, calculating the degree of bridge structural deterioration, and assessing the remaining life of the bridge in conjunction with the damage propagation rate is as follows:

[0153] Step S410: Obtain the geometric topology of the bridge structure and the mesh size Δp of the optimized sensor network;

[0154] Step S420: Dynamically adjust the local mesh size according to the intensity of the acoustic emission signal in the multimodal information. When the intensity of the acoustic emission signal of the mesh cell in the optimized sensor network is greater than the upper limit of the intensity threshold, adjust the local mesh size Δp' to half of the original mesh size; when the intensity of the acoustic emission signal of the mesh cell in the optimized sensor network is less than the lower limit of the intensity threshold, adjust the local mesh size Δp' to twice the original mesh size.

[0155] The values ​​of the upper and lower limits of the strength threshold are set by those skilled in the art based on experience.

[0156] Step S430: Define the acoustic wave propagation path compensation term Δs, calculate the difference between the recorded arrival time of the acoustic emission signal and the acoustic wave propagation path compensation term, and obtain the corrected arrival time of the acoustic emission signal s'.

[0157] The calculation formula of the sound wave propagation path compensation term is: Wherein, d j and v j are the propagation distance and speed of the acoustic emission signal on the jth segment of the propagation path, and J is the number of propagation path segments;

[0158] The calculation formula of the modified acoustic emission signal arrival time is: s' = s-Δs, wherein s is the recorded acoustic emission signal arrival time;

[0159] Step S440: preliminary positioning of the acoustic emission signal on the grid size Δp to obtain a suspected damage area, and obtaining a damage positioning result in the local grid of the suspected damage area;

[0160] The preliminary positioning uses an acoustic emission positioning algorithm, preferably, a TDOA algorithm is selected to calculate the distance between the sound source and multiple sensors, a distance difference equation set is constructed through the time difference of the acoustic emission signals received by multiple sensors, the sound source coordinates are solved, and a preliminary positioning result is obtained;

[0161] The damage positioning result in the local grid of the suspected damage area refers to selecting a suspected damage area based on the preliminary positioning, positioning on the corresponding local grid, obtaining more time difference of the acoustic emission signals, and combining the sound wave propagation path compensation term to obtain the accurate position of the damage;

[0162] The specific method for preliminary positioning of the acoustic emission signal on the grid size Δp to obtain a suspected damage area is:

[0163] Step S441: assuming that the sound source position is (xi, yi, zi), the position of the nth sensor is (xni, yni, zni), and the sound wave propagation speed is vi;

[0164] Step S442: calculating the distance dni from the sound source to the nth sensor, and calculating the time tni at which the acoustic emission signal arrives at the nth sensor;

[0165] The calculation formula of the time tni at which the acoustic emission signal arrives at the nth sensor is: Wherein, t0 is the emission time of the acoustic emission signal;

[0166] Step S443: for any two sensors ni and nj, calculating the time difference Δtij at which they receive the acoustic emission signal;

[0167] The calculation formula of the time difference is:

[0168] Step S444: Substitute the distances into the calculation formula of the time difference and solve the time difference equation set to obtain the sound source position as the preliminary positioning result and obtain the suspicious damage area;

[0169] The time difference equation set is: dn2-dn1=vi x At21, dn3-dn1=vi x At31,..., dnN-dn1=vi x AtN1, wherein dnN is the distance of the acoustic emission signal reaching the nthN sensor, nN is the number of sensors, and AtN1 represents the time difference between the sensors n1 and nN receiving the acoustic emission signal;

[0170] The solving method for solving the time difference equation set is selected from Newton iteration method or gradient descent method;

[0171] The specific method for obtaining the damage positioning result in the local grid of the suspicious damage area is:

[0172] Step S445: Assuming that M sensors are arranged on the local grid, the position of the mth sensor is (x mi ,y mi ,z mi ), for any two sensors mi and mj, they form a hyperboloid with the sound source, satisfying the hyperboloid equation:

[0173] Step S446: Combine the sound wave propagation path compensation term As to modify the hyperboloid equation to: dmi-dmj=As x vi x At ij ;

[0174] Step S447: Select multiple pairs of sensors, construct multiple hyperboloid equations, form an equation set, and solve the equation set to obtain the accurate position of the sound source as the damage positioning result;

[0175] Step S450: According to the damage positioning result, the number of damage points is counted, the damage density l af is calculated, and the bridge structure degradation degree is calculated according to the time delay and energy attenuation of the acoustic emission signal in the bridge structure;

[0176] The calculation formula of the bridge structure degradation degree is: DI=w1 x l af +w2 x AtI+w3 x EI, wherein AtI is the time delay, EI is the energy attenuation, w1, w2, and w3 are weight coefficients of the damage density, the time delay, and the energy attenuation, respectively, and the sum of the weight coefficients is 1;

[0177] The weight coefficients of the damage density, the time delay, and the energy attenuation are set by a person skilled in the art according to experience.

[0178] Step S460: calculating the remaining life Lr of the bridge structure according to the bridge structure deterioration degree and the damage propagation rate;

[0179] The calculation formula of the remaining life is: Wherein, DI max is the critical threshold of the bridge structure deterioration degree, indicating the deterioration degree when the bridge structure completely fails;

[0180] The critical threshold of the bridge structure deterioration degree is determined by the person skilled in the art according to the actual situation according to the design and construction of the bridge.

[0181] Figure 3 The bridge structure remaining life evaluation method flowchart provided in the present application is shown in the following.

[0182] Step S500: constructing a maintenance strategy knowledge base, and generating a maintenance strategy set according to the bridge damage category, the remaining life and the bridge structure deterioration degree;

[0183] The specific method for constructing the maintenance strategy knowledge base and generating the maintenance strategy set according to the bridge damage category, the remaining life and the bridge structure deterioration degree is as follows:

[0184] Step S510: collecting expert knowledge in the field of bridge maintenance, including the corresponding relationship between the damage category, the bridge structure deterioration degree, the remaining life and the maintenance strategy, and constructing the maintenance strategy knowledge base;

[0185] The rule in the maintenance strategy knowledge base is as follows: IF damage category = X AND bridge structure deterioration degree = Y AND remaining life = Z THEN maintenance strategy = S;

[0186] Step S520: taking the bridge damage category, the remaining life and the bridge structure deterioration degree as inputs, matching the feasible maintenance strategy in the maintenance strategy knowledge base, generating a maintenance strategy set, and sending the maintenance strategy set to the maintenance personnel.

[0187] Embodiment 2

[0188] As shown in the following, the multi-modal information fusion steel-UHPC composite bridge dynamic maintenance system provided in the present application includes: Figure 4

[0189] The multi-modal information acquisition module obtains an optimized sensor network based on a sensor layout optimization model, a sensor collaborative layout optimization model and a sensor number dynamic adjustment method, and monitors multi-modal information;

[0190] The damage category output module inputs the multi-modal information based on the pre-trained damage category recognition model, and outputs the damage category;

[0191] ​a damage propagation calculation module, configured to predict bridge damage features at adjacent time steps based on the damage category and using a pre-trained bridge damage evolution model, and to calculate a damage propagation rate;

[0192] a remaining life assessment module, configured to perform local grid dynamic division on the bridge based on the optimized sensor network, to obtain damage positioning results and calculate a bridge structure degradation degree, and to assess the remaining life of the bridge in combination with the damage propagation rate;

[0193] a maintenance strategy generation module, configured to construct a maintenance strategy knowledge base and to generate a maintenance strategy set based on the bridge damage category, the remaining life and the bridge structure degradation degree.

[0194] Embodiment 3

[0195] The application provides an electronic device, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform steps in a steel-UHPC combined bridge dynamic maintenance method based on multi-modal information fusion.

[0196] The application provides a readable storage medium. The readable storage medium stores computer readable instructions. When the computer readable instructions are executed by a processor, a steel-UHPC combined bridge dynamic maintenance method based on multi-modal information fusion according to the embodiments of the application can be executed. The storage medium includes but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory and cache memory, etc. The non-volatile memory may, for example, include read-only memory, hard disk, flash memory, etc.

[0197] In addition, according to the embodiments of the application, the processes described above can be implemented as a computer software program. For example, the application provides a non-transitory machine-readable storage medium storing machine-readable instructions executable by a processor to execute instructions corresponding to the method steps provided by the application, for example: obtaining an optimized sensor network based on a sensor layout optimization model, a sensor collaborative layout optimization model and a sensor number dynamic adjustment method, and monitoring multi-modal information; inputting multi-modal information based on a pre-trained damage category identification model, and outputting a damage category; predicting bridge damage features at adjacent time steps based on the damage category and using a pre-trained bridge damage evolution model, and calculating a damage propagation rate; performing local grid dynamic division on the bridge based on the optimized sensor network, obtaining damage positioning results and calculating a bridge structure degradation degree, and assessing the remaining life of the bridge in combination with the damage propagation rate; constructing a maintenance strategy knowledge base and generating a maintenance strategy set based on the bridge damage category, the remaining life and the bridge structure degradation degree. When the computer program is executed by a central processing unit, the above functions defined in the method of the application are executed.

[0198] The method and system of the present application can be implemented in various ways. The above-mentioned sequence of steps for the method is only for illustration, and the method steps of the present application are not limited to the above specifically described sequence, unless otherwise specifically stated. Furthermore, in some embodiments, the present application can also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the method according to the present application. Thus, the present application also covers the recording medium storing the programs for executing the method according to the present application.

[0199] In addition, the part of the above technical solutions provided in the embodiments of the present application which is consistent with the implementation principle of the corresponding technical solutions in the prior art is not described in detail, so as not to be too verbose.

[0200] The specific embodiments described above are further explained in detail with reference to the accompanying drawings. It should be understood that the foregoing description is only specific embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A dynamic maintenance method for steel-UHPC composite bridges based on multimodal information fusion, characterized in that, include: Based on the sensor deployment optimization model, the sensor collaborative deployment optimization model, and the sensor quantity dynamic adjustment method, an optimized sensor network is obtained to monitor multimodal information; Based on a pre-trained damage category recognition model, inputting multimodal information, outputting damage category; Based on the damage category, a pre-trained bridge damage evolution model is used to predict the bridge damage characteristics at adjacent time steps and calculate the damage propagation rate. Based on the optimized sensor network, the bridge is dynamically divided into local grids to obtain damage location results and calculate the degree of bridge structural deterioration. The remaining life of the bridge is then assessed in conjunction with the damage propagation rate. Construct a maintenance strategy knowledge base to generate a set of maintenance strategies based on bridge damage type, remaining lifespan, and degree of bridge structural deterioration. The specific method for obtaining an optimized sensor network and monitoring multimodal information based on the sensor deployment optimization model, the sensor collaborative deployment optimization model, and the dynamic adjustment method for the number of sensors is as follows: Define the damage localization accuracy requirement matrix; Define a sensor deployment decision matrix to represent the number of sensors deployed in each area; With minimizing sensor deployment cost as the objective function and the number of sensors as the optimization variable, the contribution constraint of sensor positioning accuracy is defined, a sensor deployment optimization model is constructed, and the initial optimal number of sensors for each area is obtained through optimization algorithm. Define a damage localization result matrix to represent the damage density obtained for each damage localization in each region; Design a method for dynamically adjusting the number of sensors, and update the number of sensors based on the relationship between the damage density and the preset upper and lower limits of the damage density threshold. Define a heterogeneous sensor collaborative decision matrix to represent the number of different types of sensor pairs deployed simultaneously; The objective function is to maximize the contribution of sensor cooperative localization. The number of sensor pairs deployed simultaneously is used as the optimization variable. The sensor pair number constraint is defined, and a sensor cooperative deployment optimization model is established. The optimal number of sensor pairs is obtained by solving the optimization algorithm. Based on the optimal number of sensors, the dynamic adjustment method for the number of sensors, and the optimal number of sensor pairs, an optimized sensor network is obtained. Multimodal information is collected based on the sensor network, including acoustic emission signals, vibration signals, strain signals, and image data.

2. The dynamic maintenance method for steel-UHPC composite bridges based on multimodal information fusion as described in claim 1, characterized in that, The method for dynamically adjusting the number of sensors is as follows: the sensor deployment decision matrix is ​​dynamically adjusted according to the damage location result matrix, and an upper limit and a lower limit of the damage density threshold are preset. For each region, when the damage density of the sensors in the region is greater than the upper limit of the damage density threshold, the number of sensors in the region is updated. The updated number of sensors is the sum of the current number of sensors in the region and the sensor number adjustment step size. When the damage density of sensors in the region is less than the lower limit of the damage density threshold, the difference between the current number of sensors in the region and the sensor number adjustment step size is calculated. The difference is compared with the lower limit of the number of sensors, and the maximum value between the two is selected as the updated number of sensors.

3. The dynamic maintenance method for steel-UHPC composite bridges based on multimodal information fusion as described in claim 2, characterized in that, The specific method for the pre-trained damage category recognition model to take multimodal information as input and output damage category is as follows: A damage category recognition model is constructed, which includes a feature extraction module, an attention fusion module, a deep belief network module, and a loss classification module. Multimodal information is preprocessed, and multimodal features are extracted based on the feature extraction module; The weights of the multimodal features are calculated using an attention mechanism, and the weighted multimodal features are fused based on the attention fusion module to construct a multimodal feature vector. Multimodal feature vectors are input into a deep belief network module to learn high-level feature representations. ; The high-level feature representation is input into the loss classification module. The Softmax classifier is used to predict the probability of damage category on the high-level feature representation. The probability of belonging to each damage category is output. The cross-entropy loss function is used to calculate the difference between the predicted damage category and the true damage category. The model parameters are optimized through the backpropagation algorithm to obtain the trained damage category recognition model. The trained damage category recognition model is used to predict the probability that the current multimodal information belongs to each damage category, and the damage category with the highest probability value is taken as the damage category of the current multimodal information; the damage categories include crack damage, steel corrosion damage, concrete carbonation damage, and fatigue accumulation damage.

4. The dynamic maintenance method for steel-UHPC composite bridges based on multimodal information fusion as described in claim 3, characterized in that, The specific method for predicting bridge damage characteristics at adjacent time steps based on damage categories and calculating the damage propagation rate using a pre-trained bridge damage evolution model is as follows: The bridge damage data corresponding to the multimodal information is measured, and the bridge damage data is grouped based on the different damage categories output. For each damage category, a time-varying damage feature evolution sequence is extracted from the bridge damage data; the damage features include crack width, crack length, and damage area. A bridge damage evolution model is constructed, which includes an input layer, an LSTM hidden layer, and an output layer. The input layer is used to receive the damage feature evolution sequence. One damage feature is input at each time step. The LSTM hidden layer is used to encode the damage feature at the current time step, and the output of the LSTM hidden layer is mapped to the damage feature prediction value at the next time step. A loss function is designed based on mean squared error and Paris's law regularization term. The goal is to train a bridge damage evolution model by minimizing the value of the loss function. By using a trained bridge damage evolution model to predict damage characteristics, the changes in damage characteristics at adjacent time steps can be obtained. Calculate the damage propagation rate .

5. The dynamic maintenance method for steel-UHPC composite bridges based on multimodal information fusion as described in claim 4, characterized in that, The specific method for training the bridge damage evolution model is as follows: The loss function is designed based on mean squared error and Paris's law regularization term, and the goal is to minimize the value of the loss function. The damage feature evolution sequence samples are obtained and divided into subsequences of fixed length Z. For each subsequence, the damage features of the first Z-1 time steps are used as inputs and the damage features of the last time step are used as prediction targets. The inputs of all subsequences and their corresponding prediction targets are combined into training sample pairs. The difference between the true and predicted values ​​of damage features is calculated using mean squared error. A Paris law regularization term is introduced, and the final loss function is obtained based on the weighted sum of the mean squared error and the Paris law regularization term. By minimizing the value of the loss function as the training objective, the model parameters are optimized to obtain a well-trained bridge damage evolution model.

6. The dynamic maintenance method for steel-UHPC composite bridges based on multimodal information fusion as described in claim 5, characterized in that, The specific method for dynamically dividing the bridge into local meshes based on an optimized sensor network, obtaining damage location results, calculating the degree of bridge structural deterioration, and assessing the remaining life of the bridge in conjunction with the damage propagation rate is as follows: Obtain the geometric topology of the bridge structure and the mesh size Δp of the optimized sensor network; The local mesh size is dynamically adjusted based on the intensity of the acoustic emission signal in the multimodal information. When the intensity of the acoustic emission signal of a mesh cell in the optimized sensor network exceeds the upper limit of the intensity threshold, the local mesh size is adjusted. The local mesh size is adjusted to half of the original mesh size; when the intensity of the acoustic emission signal of a mesh cell in the optimized sensor network is less than the lower limit of the intensity threshold, the local mesh size is adjusted. It is twice the size of the original grid. Define sound wave propagation path compensation term The difference between the recorded arrival time of the acoustic emission signal and the acoustic wave propagation path compensation term is calculated to obtain the corrected arrival time of the acoustic emission signal. ; Preliminary localization of acoustic emission signals is performed on the grid size Δp to obtain suspected damage areas, and damage localization results are obtained within the local grid of the suspected damage areas. The number of damage points is counted based on the damage localization results, and the damage density is calculated. The degree of bridge structure deterioration is calculated based on the time delay and energy attenuation of acoustic emission signals propagating in the bridge structure. The remaining life Lr of the bridge structure is calculated based on the degree of deterioration and the rate of damage propagation.

7. The dynamic maintenance method for steel-UHPC composite bridges based on multimodal information fusion as described in claim 6, characterized in that, The specific method for initially locating the acoustic emission signal on the grid size Δp to obtain the suspected damage area is as follows: Assuming the location of the sound source is The position of the ni-th sensor is The speed of sound wave propagation is vi; Calculate the distance dni from the sound source to the ni-th sensor, and calculate the time tni for the acoustic emission signal to reach the ni-th sensor; For any two sensors ni and nj, calculate the time difference Δtij between their receipt of acoustic emission signals; Substituting the distance into the time difference calculation formula and solving the system of time difference equations yields a set of time difference equations. Solving the system of time difference equations provides the location of the sound source as a preliminary localization result, thus identifying the suspected damage area.

8. A multimodal information fusion-based dynamic maintenance system for steel-UHPC composite bridges, used to implement the multimodal information fusion-based dynamic maintenance method for steel-UHPC composite bridges as described in any one of claims 1-7, characterized in that, include: The multimodal information acquisition module, based on the sensor deployment optimization model, the sensor collaborative deployment optimization model, and the sensor number dynamic adjustment method, obtains an optimized sensor network to monitor multimodal information; The damage category output module, based on a pre-trained damage category recognition model, takes multimodal information as input and outputs the damage category. The damage propagation calculation module, based on the damage category, uses a pre-trained bridge damage evolution model to predict the bridge damage characteristics at adjacent time steps and calculates the damage propagation rate. The remaining life assessment module dynamically divides the bridge into local grids based on an optimized sensor network, obtains damage location results, calculates the degree of bridge structural deterioration, and assesses the remaining life of the bridge in combination with the damage propagation rate. The maintenance strategy generation module is used to build a maintenance strategy knowledge base, generating a set of maintenance strategies based on the bridge damage type, remaining life, and degree of bridge structural deterioration.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the dynamic maintenance method for steel-UHPC composite bridges based on multimodal information fusion as described in any one of claims 1-7.

Images