Bridge structure safety monitoring method and system based on artificial intelligence

Abstract

The invention discloses a bridge structure safety monitoring method and system based on artificial intelligence. The method comprises the steps of data acquisition, data preprocessing, construction of a first monitoring module, construction of a second monitoring module, construction of a bridge safety monitoring model and bridge structure safety monitoring. The invention relates to the technical field of bridge structure safety monitoring, in particular to a bridge structure safety monitoring method and system based on artificial intelligence. A data preprocessing method of data cleaning, data annotation, data coding, data normalization, image expansion and data set segmentation is adopted; a graph attention gating time convolutional network is adopted to carry out bridge sensor data analysis, and a space-time dependency relationship and nonlinear characteristics in sensor data can be dynamically captured; two different modules are adopted to process sensor data and image data respectively, multi-dimensional information of structural health is effectively captured, and a more comprehensive monitoring effect is provided.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge structure safety monitoring, and in particular to an artificial intelligence-based bridge structure safety monitoring method and system. Background Art

[0002] Bridge structure safety monitoring is to monitor the structural status, load response and environmental impact of bridges in real time or regularly through advanced technical means and equipment to ensure their safety and stability. Through continuous monitoring and evaluation, potential safety hazards can be discovered in time, accidents can be prevented, and traffic safety can be protected; at the same time, monitoring data can help optimize bridge maintenance and management, extend service life, and provide a scientific basis for bridge design and construction.

[0003] However, traditional bridge structure safety monitoring methods have limited performance in dealing with nonlinear relationships and time-series dependencies of safety monitoring data, and are difficult to cope with large-scale bridge sensor data, heterogeneous characteristics, and long-series monitoring data, resulting in poor performance of the monitoring model. Traditional bridge structure safety monitoring methods usually use a single visual inspection or sensor monitoring, which makes it difficult to fully utilize multimodal bridge safety monitoring data and multi-dimensional bridge structure safety information, resulting in the prediction results being easily affected by the environment and not accurate enough. Summary of the invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a bridge structure safety monitoring method and system based on artificial intelligence. The traditional bridge structure safety monitoring method has limited performance in processing the nonlinear relationship and time series dependency of safety monitoring data, and is difficult to deal with large-scale bridge sensor data, heterogeneous characteristics and long time series monitoring data, resulting in poor performance of the monitoring model. The scheme creatively adopts a graph attention gated temporal convolutional network to analyze bridge sensor data, which can dynamically capture the spatiotemporal dependency and nonlinear characteristics in the sensor data, adapt to the high dimensionality and complexity of the data, and improve the accuracy and robustness of the prediction; the traditional bridge structure safety monitoring method usually uses a single visual inspection or sensor monitoring, which is difficult to make full use of multimodal bridge safety monitoring data and multi-dimensional bridge structure safety information, resulting in the prediction results being susceptible to environmental influences and not accurate enough. The scheme creatively adopts two different modules to process sensor data and image data respectively, effectively captures the multi-dimensional information of structural health, and weighted sums the prediction results of the two models, which can comprehensively consider the advantages and disadvantages of different data sources, thereby providing a more comprehensive monitoring effect.

[0005] The technical solution adopted by the present invention is as follows: The bridge structure safety monitoring method based on artificial intelligence provided by the present invention comprises the following steps:

[0006] Step S1: data collection;

[0007] Step S2: data preprocessing;

[0008] Step S3: constructing the first monitoring module;

[0009] Step S4: constructing the second monitoring module;

[0010] Step S5: constructing a bridge safety monitoring model;

[0011] Step S6: bridge structure safety monitoring.

[0012] Furthermore, in step S1, the data collection is used to collect the original data required for monitoring the safety of the bridge structure, specifically, to obtain the original data set of the bridge condition through data collection;

[0013] The original data set of bridge conditions specifically includes an original image data set and an original sensor data set. The bridge image data set specifically includes an overall structure image of the bridge, a local close-up image of the bridge, an image of a bridge connection node, and a real-time monitoring image of the bridge. The bridge sensor data set specifically includes bridge displacement data, bridge strain gauge data, bridge acceleration data, bridge temperature data, and bridge pressure data.

[0014] Furthermore, in step S2, the data preprocessing is used to preprocess the collected raw data, and specifically includes the following steps:

[0015] Step S21: sensor data preprocessing, specifically, performing data cleaning, data labeling, data encoding, data normalization and data set segmentation on the original sensor data set to obtain a sensor training set and a sensor test set;

[0016] Step S22: image data preprocessing, specifically performing image cleaning, image expansion, image annotation and data set segmentation on the original image data set to obtain an image training set and an image test set;

[0017] Step S23: preprocessing, specifically obtaining a sensor training set and a sensor test set by preprocessing the sensor data, and obtaining an image training set and an image test set by preprocessing the image data.

[0018] Further, in step S3, the first monitoring module is constructed to construct a module for processing bridge sensor data, wherein the first monitoring module specifically includes an enhanced graph attention layer, a gated temporal convolution layer, and an output layer;

[0019] The construction of the first monitoring module specifically includes the following steps:

[0020] Step S31: constructing graph structure data, specifically, constructing graph structure data by taking features as nodes and establishing edges based on the correlation between features;

[0021] Step S32: Enhanced graph attention layer construction, the steps include:

[0022] Step S321: Calculate the attention score, which is used to determine the attention score between nodes. Specifically, the linear mapping in the self-attention mechanism is replaced by a one-dimensional convolution operation, and the attention score is calculated. The formula used for calculating the attention score is as follows:

[0023] ;

[0024] In the formula, represents the attention score between the i-th node and the j-th node in the l-th enhanced graph attention layer, represents the convolution kernel used for convolution operation on the i-th node of the l-th enhanced graph attention layer, represents the convolution kernel used to perform convolution operation on the jth node of the lth enhanced graph attention layer, represents the feature vector of the i-th node in the l-th enhanced graph attention layer, represents the feature vector of the jth node in the lth enhanced graph attention layer, represents the learnable parameter matrix used by the l-th enhanced graph attention layer to adjust the output of the convolution operation, Represents a one-dimensional convolution operation;

[0025] Step S322: Calculate the attention weight, which is used to determine the attention weight between nodes. The formula used is as follows:

[0026] ;

[0027] In the formula, represents the attention weight between the i-th node and the j-th node in the l-th enhanced graph attention layer, represents the softmax normalized function of the i-th node and neighboring nodes in the l-th enhanced graph attention layer;

[0028] Step S323: Update the node, which is used to update the node feature vector. The formula used is as follows:

[0029] ;

[0030] In the formula, represents the feature vector of the i-th node in the l+1-th enhanced graph attention layer, represents the set of neighbor nodes of the ith node in the lth enhanced graph attention layer, represents the weight matrix of the jth node of the lth enhanced graph attention layer;

[0031] Step S324: constructing an enhanced graph attention layer, specifically, constructing the enhanced graph attention layer by calculating the attention score, calculating the attention weight, and updating the node to obtain the enhanced graph attention layer;

[0032] Step S33: constructing a gated temporal convolutional layer, including:

[0033] Step S331: constructing a parallel temporal convolution layer, specifically constructing two identical temporal convolution layers using an extended causal convolution operation, wherein the extended causal convolution operation uses the following formula:

[0034] ;

[0035] In the formula, represents the output function of the temporal convolutional layer, represents the convolution kernel operation function, represents the input function of the temporal convolution layer, t represents the time step, Lt represents the length of the convolution kernel, lt represents the convolution kernel index, and d represents the delay step;

[0036] Step S332: Design a gating mechanism to add a gating mechanism to the temporal convolutional layer. The formula used is as follows:

[0037] ;

[0038] Where Go represents the output of the gating mechanism, represents the hyperbolic tangent function, represents the sigmoid activation function, represents the output of the first temporal convolutional layer, represents the output of the second temporal convolutional layer, Represents the output of the first time convolution layer The weight matrix for linear transformation, Represents the output of the second temporal convolutional layer The weight matrix for linear transformation, Represents the output used to adjust the first temporal convolutional layer The bias term of the linear transformation, Represents the output used to adjust the second temporal convolutional layer The bias term of the linear transformation, Represents element-by-element multiplication operation;

[0039] Step S333: constructing a gated temporal convolutional layer, specifically, constructing the gated temporal convolutional layer by constructing the parallel temporal convolutional layer and designing the gating mechanism, to obtain the gated temporal convolutional layer;

[0040] Step S34: Establishing residual connections and skip connections, specifically constructing residual connection units and skip connection units, wherein the residual connection units are used to connect the input and output of each hidden layer of the module and serve as the input of the next hidden layer, and the skip connection units are used to connect the input of all hidden layers and the output of the last hidden layer and serve as the input of the output layer;

[0041] Step S35: Calculate module output, the formula used is as follows:

[0042] ;

[0043] In the formula, Indicates the output of the first monitoring module, represents the softmax function, W represents the output weight of the output layer of the first monitoring module, p represents the output bias term of the output layer of the first monitoring module, and z represents the input of the output layer of the first monitoring module;

[0044] Step S36: Design a loss function, the formula used is as follows:

[0045] ;

[0046] In the formula, Loss represents the loss function value of the first monitoring module, Q represents the total number of samples, represents the true label of the qth sample, represents the predicted value of the first monitoring module for the qth sample;

[0047] Step S37: constructing a module and training it, specifically, constructing a first monitoring module through the graph structure data construction, the enhanced graph attention layer construction, the gated time convolution layer construction, the establishment of residual connections and jump connections, the calculation model output and the design of the loss function, and performing module training based on the sensor training set, verifying the module performance based on the sensor test set, and obtaining the first monitoring module.

[0048] Further, in step S4, the second monitoring module is constructed to construct a module for processing bridge image data, and the second monitoring module specifically includes a residual network submodule, a visual transformer submodule, a feature fusion submodule and an output submodule;

[0049] The second monitoring module is constructed, specifically comprising the following steps:

[0050] Step S41: feature extraction, specifically using the residual network submodule and the visual transformer submodule to perform feature extraction

[0051] Step S42: constructing a feature fusion submodule, the steps include:

[0052] Step S421: Preliminary feature processing, the formula used is as follows:

[0053] ;

[0054] In the formula, represents the flattening function, represents the GELU activation function, represents the batch normalization operation function, represents the convolution operation function, represents the features extracted by the residual network submodule, represents the features extracted by the visual transformer submodule, represents the features extracted by the residual network submodule after preliminary processing, represents the features extracted by the visual transformer submodule after preliminary processing;

[0055] Step S422: Design a self-attention mechanism. The formula used is as follows:

[0056] ;

[0057] Where Qu represents the query vector, Ke represents the key vector, and Va represents the value vector. represents the weight matrix used to calculate the query vector, represents the weight matrix used to calculate the key vector, represents the weight matrix used to calculate the value vector, represents the layer normalization operation function, S represents the attention score, T represents the transposition operation, represents the dimension of the key vector;

[0058] Step S423: Calculate the submodule output using the following formula:

[0059] ;

[0060] In the formula, Os represents the output of the feature fusion submodule, represents the weight matrix used to control the linear transformation of the attention score, represents the bias term used to control the linear transformation of the attention score, represents the pooling operation function, Represents element-by-element addition operation;

[0061] Step S424: submodule construction, specifically, constructing a feature fusion submodule through the feature preliminary processing, the designed self-attention mechanism and the calculation submodule output, to obtain the feature fusion submodule;

[0062] Step S43: Calculate module output, the formula used is as follows:

[0063] ;

[0064] In the formula, Indicates the output of the second monitoring module, represents the output weight of the second monitoring module output submodule, represents the output bias item of the output submodule of the second monitoring module;

[0065] Step S44: designing a loss function, the steps include:

[0066] Step S441: Calculate the attention matrix. The formula used is as follows:

[0067] ;

[0068] Where Am represents the attention matrix;

[0069] Step S442: Calculate the cross-transposed attention matrix, using the following formula:

[0070] ;

[0071] Where Ca represents the cross-transposed attention matrix;

[0072] Step S443: Calculate the total loss using the following formula:

[0073] ;

[0074] In the formula, represents the cross entropy loss value of the second monitoring module, represents the cross entropy loss value based on the cross-transposed attention matrix of the second monitoring module, N represents the total number of image samples, represents the true label of the nth image sample, represents the predicted value of the second monitoring module for the nth image sample, M represents the total number of image sample categories, represents the weight of the mth category, Ta represents the attention matrix generated based on the true label, represents the element of the nth row and mth column of the attention matrix generated based on the true label, represents the element of the cross-transposed attention matrix in the nth row and mth column, Indicates the total loss value of the second monitoring module;

[0075] Step S45: constructing a module and training it, specifically, constructing a second monitoring module through the feature extraction, constructing a feature fusion submodule, the calculation module output and the design loss function, and performing module training based on the image training set, and verifying the module performance based on the image test set to obtain a second monitoring module.

[0076] Further, in step S5, the bridge safety monitoring model is constructed, specifically, a bridge safety monitoring model is constructed, and the bridge safety monitoring model specifically includes a first monitoring module, a second monitoring module and a comprehensive output module. The comprehensive output module is used to integrate the output of the first monitoring module and the output of the second monitoring module, and output the prediction result of the model, specifically, by performing weighted summation on the output of the first monitoring module and the output of the second monitoring module to obtain comprehensive bridge safety reference data.

[0077] Furthermore, in step S6, the bridge structure safety monitoring is specifically to analyze the safety status of the bridge structure based on the comprehensive bridge safety reference data, thereby realizing the monitoring of the bridge structure safety.

[0078] The artificial intelligence-based bridge structure safety monitoring system provided by the present invention comprises a data acquisition module, a data preprocessing module, a first monitoring module construction module, a second monitoring module construction module, a bridge safety monitoring model construction module and a bridge structure safety monitoring module;

[0079] The data acquisition module is used for data acquisition, and obtains an original data set of bridge conditions through data acquisition, and sends the original data set of bridge conditions to the data preprocessing module;

[0080] The data preprocessing module is used for data preprocessing, and obtains a sensor training set, a sensor test set, an image training set and an image test set through data preprocessing, and sends the sensor training set and the sensor test set to the first monitoring module construction module, and sends the image training set and the image test set to the second monitoring module construction module;

[0081] The first monitoring module construction module is used to construct the first monitoring module, obtain the first monitoring module by constructing the first monitoring module, and send the first monitoring module to the bridge safety monitoring model construction module;

[0082] The second monitoring module construction module is used to construct the second monitoring module, obtain the second monitoring module by constructing the second monitoring module, and send the second monitoring module to the bridge safety monitoring model construction module;

[0083] The bridge safety monitoring model construction module is used to construct a bridge safety monitoring model, obtain comprehensive bridge safety reference data by constructing the bridge safety monitoring model, and send the comprehensive bridge safety reference data to the bridge structure safety monitoring module;

[0084] The bridge structure safety monitoring module is used for bridge structure safety monitoring, and realizes monitoring of bridge structure safety by analyzing the safety status of the bridge structure based on the comprehensive bridge safety reference data.

[0085] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0086] (1) Traditional bridge structure safety monitoring methods have limited performance in dealing with nonlinear relationships and time series dependencies of safety monitoring data, and are difficult to handle with large-scale bridge sensor data, heterogeneous characteristics, and long time series monitoring data, resulting in poor performance of the monitoring model. This scheme creatively uses a graph attention gated temporal convolutional network to analyze bridge sensor data. It can dynamically capture the spatiotemporal dependencies and nonlinear characteristics in sensor data, adapt to the high dimensionality and complexity of the data, and improve the accuracy and robustness of the prediction.

[0087] (2) Traditional bridge structure safety monitoring methods usually use a single visual inspection or sensor monitoring, which makes it difficult to fully utilize multimodal bridge safety monitoring data and multi-dimensional bridge structure safety information, resulting in the prediction results being easily affected by the environment and not accurate enough. This solution creatively uses two different modules to process sensor data and image data respectively, effectively capturing the multi-dimensional information of structural health, and performing a weighted summation of the prediction results of the two models, which can comprehensively consider the advantages and disadvantages of different data sources, thereby providing a more comprehensive monitoring effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 A schematic diagram of the flow chart of the bridge structure safety monitoring method based on artificial intelligence provided by the present invention;

[0089] Figure 2 A schematic diagram of a module of a bridge structure safety monitoring system based on artificial intelligence provided by the present invention;

[0090] Figure 3 A schematic diagram of the process of constructing the first monitoring module in step S3;

[0091] Figure 4 A schematic diagram of the process of constructing the second monitoring module in step S4;

[0092] Figure 5 This is a schematic diagram of the structure of the first monitoring module in step S3 of constructing the first monitoring module.

[0093] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0094] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0095] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0096] Example 1, see Figure 1 The technical solution adopted by the present invention is as follows: The bridge structure safety monitoring method based on artificial intelligence provided by the present invention comprises the following steps:

[0097] Step S1: data collection;

[0098] Step S2: data preprocessing;

[0099] Step S3: constructing the first monitoring module;

[0100] Step S4: constructing the second monitoring module;

[0101] Step S5: constructing a bridge safety monitoring model;

[0102] Step S6: bridge structure safety monitoring.

[0103] Example 2, see Figure 1 and Figure 2 In step S1, the data acquisition is used to acquire the original data required for monitoring the safety of the bridge structure, specifically, to obtain the original data set of the bridge condition through data acquisition;

[0104] The original data set of bridge conditions specifically includes an original image data set and an original sensor data set. The bridge image data specifically includes an image of the overall structure of the bridge, a close-up image of a part of the bridge, an image of a bridge connection node, and a real-time monitoring image of the bridge. The bridge sensor data specifically includes bridge displacement data, bridge strain gauge data, bridge acceleration data, bridge temperature data, and bridge pressure data.

[0105] Example 3, see Figure 1 and Figure 2This embodiment is based on the above embodiment. In step S2, the data preprocessing is used to preprocess the collected raw data, and specifically includes the following steps:

[0106] Step S21: Sensor data preprocessing, the steps include:

[0107] Step S211: sensor data cleaning, which is used to clean the original sensor data, specifically to remove missing values ​​and duplicate values ​​in the original sensor data set to obtain a sensor rough processing data set;

[0108] Step S212: sensor data labeling, which is used to label the sensor rough-processed data, specifically labeling the data in the sensor rough-processed data set as safe, low-risk, medium-risk and high-risk, and using them as data labels to obtain a sensor labeled data set;

[0109] Step S213: sensor data encoding, which is used to encode the sensor tag data, specifically, using a one-hot encoding method to encode the sensor tag data set to obtain a sensor coded data set;

[0110] Step S214: sensor data normalization, which is used to normalize the sensor coded data, specifically, using the minimum-maximum method to normalize the sensor coded data set to obtain a sensor normalized data set;

[0111] Step S215: data set segmentation, which is used to segment the data set, specifically segmenting the sensor normalized data set to obtain a sensor training set and a sensor test set;

[0112] Step S22: image data preprocessing, the steps include:

[0113] Step S221: image cleaning, which is used to clean the original image data, specifically to remove missing values ​​and duplicate values ​​in the original image data set to obtain a roughly processed image set;

[0114] Step S222: image expansion, used to expand image data, specifically, performing random geometric transformation of the image and random brightness adjustment of the image, and expanding the coarsely processed image set to obtain an expanded image set;

[0115] Step S223: image annotation, which is used to annotate the expanded images, specifically annotating the data in the expanded image set as safe, low-risk, medium-risk and high-risk, and using them as data labels to obtain a labeled image set;

[0116] Step S224: data set segmentation, for segmenting the data set, specifically segmenting the labeled image set to obtain an image training set and an image test set;

[0117] Step S23: preprocessing, specifically obtaining a sensor training set and a sensor test set by preprocessing the sensor data, and obtaining an image training set and an image test set by preprocessing the image data.

[0118] Example 4, see Figure 1 , Figure 2 , Figure 3 and Figure 5 ,This embodiment is based on the above embodiment. In step S3, the first monitoring module is constructed to construct a module for processing bridge sensor data. The first monitoring module specifically includes an enhanced graph attention layer, a gated temporal convolution layer and an output layer;

[0119] The first monitoring module is constructed, specifically comprising the following steps:

[0120] Step S31: constructing graph structure data, specifically, constructing graph structure data by taking features as nodes and establishing edges based on the correlation between features;

[0121] Step S32: Enhanced graph attention layer construction, the steps include:

[0122] Step S321: Calculate the attention score, which is used to determine the attention score between nodes. Specifically, the linear mapping in the self-attention mechanism is replaced by a one-dimensional convolution operation, and the attention score is calculated. The formula used for calculating the attention score is as follows:

[0123] ;

[0124] In the formula, represents the attention score between the i-th node and the j-th node in the l-th enhanced graph attention layer, represents the convolution kernel used for convolution operation on the i-th node of the l-th enhanced graph attention layer, represents the convolution kernel used to perform convolution operation on the jth node of the lth enhanced graph attention layer, represents the feature vector of the i-th node in the l-th enhanced graph attention layer, represents the feature vector of the jth node in the lth enhanced graph attention layer, represents the learnable parameter matrix used by the l-th enhanced graph attention layer to adjust the output of the convolution operation, Represents a one-dimensional convolution operation;

[0125] Step S322: Calculate the attention weight, which is used to determine the attention weight between nodes. The formula used is as follows:

[0126] ;

[0127] In the formula, represents the attention weight between the i-th node and the j-th node in the l-th enhanced graph attention layer, represents the softmax normalized function of the i-th node and neighboring nodes in the l-th enhanced graph attention layer;

[0128] Step S323: Update the node, which is used to update the node feature vector. The formula used is as follows:

[0129] ;

[0130] In the formula, represents the feature vector of the i-th node in the l+1-th enhanced graph attention layer, represents the set of neighbor nodes of the ith node in the lth enhanced graph attention layer, represents the weight matrix of the jth node of the lth enhanced graph attention layer;

[0131] Step S324: constructing an enhanced graph attention layer, specifically, constructing the enhanced graph attention layer by calculating the attention score, calculating the attention weight, and updating the node to obtain the enhanced graph attention layer;

[0132] Step S33: constructing a gated temporal convolutional layer, including:

[0133] Step S331: constructing a parallel temporal convolution layer, specifically constructing two identical temporal convolution layers using an extended causal convolution operation, wherein the extended causal convolution operation uses the following formula:

[0134] ;

[0135] In the formula, represents the output function of the temporal convolutional layer, represents the convolution kernel operation function, represents the input function of the temporal convolution layer, t represents the time step, Lt represents the length of the convolution kernel, lt represents the convolution kernel index, and d represents the delay step;

[0136] Step S332: Design a gating mechanism to add a gating mechanism to the temporal convolutional layer. The formula used is as follows:

[0137] ;

[0138] Where Go represents the output of the gating mechanism, represents the hyperbolic tangent function, represents the sigmoid activation function, represents the output of the first temporal convolutional layer, represents the output of the second temporal convolutional layer, Represents the output of the first time convolution layer The weight matrix for linear transformation, Represents the output of the second temporal convolutional layer The weight matrix for linear transformation, Represents the output used to adjust the first temporal convolutional layer The bias term of the linear transformation, Represents the output used to adjust the second temporal convolutional layer The bias term of the linear transformation, Represents element-by-element multiplication operation;

[0139] Step S333: constructing a gated temporal convolutional layer, specifically, constructing the gated temporal convolutional layer by constructing the parallel temporal convolutional layer and designing the gating mechanism, to obtain the gated temporal convolutional layer;

[0140] Step S34: Establishing residual connections and skip connections, specifically constructing residual connection units and skip connection units, wherein the residual connection units are used to connect the input and output of each hidden layer of the module and serve as the input of the next hidden layer, and the skip connection units are used to connect the input of all hidden layers and the output of the last hidden layer and serve as the input of the output layer;

[0141] Step S35: Calculate module output, the formula used is as follows:

[0142] ;

[0143] In the formula, Indicates the output of the first monitoring module, represents the softmax function, W represents the output weight of the output layer of the first monitoring module, p represents the output bias term of the output layer of the first monitoring module, and z represents the input of the output layer of the first monitoring module;

[0144] Step S36: Design a loss function, the formula used is as follows:

[0145] ;

[0146] In the formula, Loss represents the loss function value of the first monitoring module, Q represents the total number of samples, represents the true label of the qth sample, represents the predicted value of the first monitoring module for the qth sample;

[0147] Step S37: constructing a module and training it, specifically, constructing a first monitoring module through the graph structure data construction, the enhanced graph attention layer construction, the gated time convolution layer construction, the establishment of residual connections and jump connections, the calculation model output and the design of the loss function, and performing module training based on the sensor training set, verifying the module performance based on the sensor test set, and obtaining the first monitoring module.

[0148] By performing the above operations, this scheme creatively uses a graph attention gated temporal convolutional network to analyze bridge sensor data, which can dynamically capture the spatiotemporal dependencies and nonlinear characteristics in sensor data, adapt to the high dimensionality and complexity of data, and improve the accuracy and robustness of prediction.

[0149] Example 5, see Figure 1 , Figure 2 and Figure 4 ,This embodiment is based on the above embodiment. In step S4, the second monitoring module is constructed to construct a module for processing bridge image data. The second monitoring module specifically includes a residual network submodule, a visual transformer submodule, a feature fusion submodule and an output submodule;

[0150] The second monitoring module is constructed, specifically comprising the following steps:

[0151] Step S41: feature extraction, specifically using the residual network submodule and the visual transformer submodule to perform feature extraction

[0152] Step S42: constructing a feature fusion submodule, the steps include:

[0153] Step S421: Preliminary feature processing, the formula used is as follows:

[0154] ;

[0155] In the formula, represents the flattening function, represents the GELU activation function, represents the batch normalization operation function, represents the convolution operation function, represents the features extracted by the residual network submodule, represents the features extracted by the visual transformer submodule, represents the features extracted by the residual network submodule after preliminary processing, represents the features extracted by the visual transformer submodule after preliminary processing;

[0156] Step S422: Design a self-attention mechanism. The formula used is as follows:

[0157] ;

[0158] Where Qu represents the query vector, Ke represents the key vector, and Va represents the value vector. represents the weight matrix used to calculate the query vector, represents the weight matrix used to calculate the key vector, represents the weight matrix used to calculate the value vector, represents the layer normalization operation function, S represents the attention score, T represents the transposition operation, represents the dimension of the key vector;

[0159] Step S423: Calculate the submodule output using the following formula:

[0160] ;

[0161] In the formula, Os represents the output of the feature fusion submodule, represents the weight matrix used to control the linear transformation of the attention score, represents the bias term used to control the linear transformation of the attention score, represents the pooling operation function, Represents element-by-element addition operation;

[0162] Step S424: submodule construction, specifically, constructing a feature fusion submodule through the feature preliminary processing, the designed self-attention mechanism and the calculation submodule output, to obtain the feature fusion submodule;

[0163] Step S43: Calculate module output, the formula used is as follows:

[0164] ;

[0165] In the formula, Indicates the output of the second monitoring module, represents the output weight of the second monitoring module output submodule, represents the output bias item of the output submodule of the second monitoring module;

[0166] Step S44: designing a loss function, the steps include:

[0167] Step S441: Calculate the attention matrix. The formula used is as follows:

[0168] ;

[0169] Where Am represents the attention matrix;

[0170] Step S442: Calculate the cross-transposed attention matrix, using the following formula:

[0171] ;

[0172] Where Ca represents the cross-transposed attention matrix;

[0173] Step S443: Calculate the total loss using the following formula:

[0174] ;

[0175] In the formula, represents the cross entropy loss value of the second monitoring module, represents the cross entropy loss value based on the cross-transposed attention matrix of the second monitoring module, N represents the total number of image samples, represents the true label of the nth image sample, represents the predicted value of the second monitoring module for the nth image sample, M represents the total number of image sample categories, represents the weight of the mth category, Ta represents the attention matrix generated based on the true label, represents the element of the nth row and mth column of the attention matrix generated based on the true label, represents the element of the cross-transposed attention matrix in the nth row and mth column, Indicates the total loss value of the second monitoring module;

[0176] Step S45: constructing a module and training it, specifically, constructing a second monitoring module through the feature extraction, constructing a feature fusion submodule, the calculation module output and the design loss function, and performing module training based on the image training set, and verifying the module performance based on the image test set to obtain a second monitoring module.

[0177] Example 6, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S5, the bridge safety monitoring model is constructed, specifically, the bridge safety monitoring model is constructed. The bridge safety monitoring model specifically includes a first monitoring module, a second monitoring module and a comprehensive output module. The comprehensive output module is used to integrate the output of the first monitoring module and the output of the second monitoring module, and output the prediction result of the model, specifically, by weighted summing the output of the first monitoring module and the output of the second monitoring module to obtain comprehensive bridge safety reference data;

[0178] The weighted sum of the output of the first monitoring module and the output of the second monitoring module is performed using the following formula:

[0179] ;

[0180] In the formula, represents the comprehensive bridge safety reference data output by the model, Represents a weight parameter used to control the output ratio of the first monitoring module and the output of the second monitoring module.

[0181] By performing the above operations, the traditional bridge structure safety monitoring method usually uses a single visual inspection or sensor monitoring, which is difficult to fully utilize multimodal bridge safety monitoring data and multi-dimensional bridge structure safety information, resulting in the prediction results being easily affected by the environment and not accurate enough. This solution creatively uses two different modules to process sensor data and image data respectively, effectively capturing the multi-dimensional information of structural health, and performing weighted summation of the prediction results of the two models, which can comprehensively consider the advantages and disadvantages of different data sources, thereby providing a more comprehensive monitoring effect.

[0182] Embodiment 7, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S6, the bridge structure safety monitoring is specifically to analyze the safety status of the bridge structure based on the comprehensive bridge safety reference data, thereby realizing the monitoring of the bridge structure safety.

[0183] Embodiment 8, see Figure 1 and Figure 2 , this embodiment is based on the above embodiment, the artificial intelligence-based bridge structure safety monitoring system provided by the present invention includes a data acquisition module, a data preprocessing module, a first monitoring module construction module, a second monitoring module construction module, a bridge safety monitoring model construction module and a bridge structure safety monitoring module;

[0184] The data acquisition module is used for data acquisition, and obtains an original data set of bridge conditions through data acquisition, and sends the original data set of bridge conditions to the data preprocessing module;

[0185] The data preprocessing module is used for data preprocessing, and obtains a sensor training set, a sensor test set, an image training set and an image test set through data preprocessing, and sends the sensor training set and the sensor test set to the first monitoring module construction module, and sends the image training set and the image test set to the second monitoring module construction module;

[0186] The first monitoring module construction module is used to construct the first monitoring module, obtain the first monitoring module by constructing the first monitoring module, and send the first monitoring module to the bridge safety monitoring model construction module;

[0187] The second monitoring module construction module is used to construct the second monitoring module, obtain the second monitoring module by constructing the second monitoring module, and send the second monitoring module to the bridge safety monitoring model construction module;

[0188] The bridge safety monitoring model construction module is used to construct a bridge safety monitoring model, obtain comprehensive bridge safety reference data by constructing the bridge safety monitoring model, and send the comprehensive bridge safety reference data to the bridge structure safety monitoring module;

[0189] The bridge structure safety monitoring module is used for bridge structure safety monitoring, and realizes monitoring of bridge structure safety by analyzing the safety status of the bridge structure based on the comprehensive bridge safety reference data.

[0190] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0191] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

[0192] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A bridge structure safety monitoring method based on artificial intelligence, characterized by: The method comprises the following steps: Step S1: data collection; Step S2: data preprocessing, preprocessing the collected raw data of bridge conditions to obtain a sensor training set, a sensor test set, an image training set and an image test set; Step S3: constructing a first monitoring module, specifically constructing a sensor data processing network to obtain a first monitoring module; Step S4: constructing a second monitoring module, specifically constructing an image data processing network to obtain a second monitoring module; Step S5: constructing a bridge safety monitoring model, specifically constructing a bridge safety monitoring model, performing weighted summation on the output of the first monitoring module and the output of the second monitoring module to obtain comprehensive bridge safety reference data; Step S6: bridge structure safety monitoring, specifically monitoring the bridge structure safety based on the comprehensive bridge safety reference data.

2. The bridge structure safety monitoring method based on artificial intelligence according to claim 1 is characterized in that: In step S3, the first monitoring module is constructed to construct a module for processing bridge sensor data, wherein the first monitoring module specifically includes an enhanced graph attention layer, a gated temporal convolution layer, and an output layer; The first monitoring module is constructed, specifically comprising the following steps: Step S31: constructing graph structure data, specifically, constructing graph structure data by taking features as nodes and establishing edges based on the correlation between features; Step S32: Enhanced graph attention layer construction, the steps include: Step S321: Calculate the attention score, which is used to determine the attention score between nodes. Specifically, the linear mapping in the self-attention mechanism is replaced by a one-dimensional convolution operation, and the attention score is calculated. The formula used for calculating the attention score is as follows: ； In the formula, represents the attention score between the i-th node and the j-th node in the l-th enhanced graph attention layer, represents the convolution kernel used for convolution operation on the i-th node of the l-th enhanced graph attention layer, represents the convolution kernel used to perform convolution operation on the jth node of the lth enhanced graph attention layer, represents the feature vector of the i-th node in the l-th enhanced graph attention layer, represents the feature vector of the jth node in the lth enhanced graph attention layer, represents the learnable parameter matrix used by the l-th enhanced graph attention layer to adjust the output of the convolution operation, Represents a one-dimensional convolution operation; Step S322: Calculate the attention weight, which is used to determine the attention weight between nodes. The formula used is as follows: ； In the formula, represents the attention weight between the i-th node and the j-th node in the l-th enhanced graph attention layer, represents the softmax normalized function of the i-th node and neighboring nodes in the l-th enhanced graph attention layer; Step S323: Update the node, which is used to update the node feature vector. The formula used is as follows: ； In the formula, represents the feature vector of the i-th node in the l+1-th enhanced graph attention layer, represents the set of neighbor nodes of the ith node in the lth enhanced graph attention layer, represents the weight matrix of the jth node in the lth enhanced graph attention layer; Step S324: constructing an enhanced graph attention layer, specifically, constructing the enhanced graph attention layer by calculating the attention score, calculating the attention weight, and updating the node to obtain the enhanced graph attention layer; Step S33: constructing a gated temporal convolutional layer, including: Step S331: constructing a parallel temporal convolution layer, specifically constructing two identical temporal convolution layers using an extended causal convolution operation, wherein the extended causal convolution operation uses the following formula: ； In the formula, represents the output function of the temporal convolutional layer, represents the convolution kernel operation function, represents the input function of the temporal convolution layer, t represents the time step, Lt represents the length of the convolution kernel, lt represents the convolution kernel index, and d represents the delay step; Step S332: Design a gating mechanism to add a gating mechanism to the temporal convolutional layer. The formula used is as follows: ； Where Go represents the output of the gating mechanism, represents the hyperbolic tangent function, represents the sigmoid activation function, represents the output of the first temporal convolutional layer, represents the output of the second temporal convolutional layer, Represents the output of the first time convolution layer The weight matrix for linear transformation, Represents the output of the second temporal convolutional layer The weight matrix for linear transformation, Represents the output used to adjust the first temporal convolutional layer The bias term of the linear transformation, Represents the output used to adjust the second temporal convolutional layer The bias term of the linear transformation, Represents element-by-element multiplication operation; Step S333: constructing a gated temporal convolutional layer, specifically, constructing the gated temporal convolutional layer by constructing the parallel temporal convolutional layer and designing the gating mechanism, to obtain the gated temporal convolutional layer; Step S34: Establishing residual connections and skip connections, specifically constructing residual connection units and skip connection units, wherein the residual connection units are used to connect the input and output of each hidden layer of the module and serve as the input of the next hidden layer, and the skip connection units are used to connect the input of all hidden layers and the output of the last hidden layer and serve as the input of the output layer; Step S35: Calculate module output, the formula used is as follows: ； In the formula, Indicates the output of the first monitoring module, represents the softmax function, W represents the output weight of the output layer of the first monitoring module, p represents the output bias term of the output layer of the first monitoring module, and z represents the input of the output layer of the first monitoring module; Step S36: Design a loss function, the formula used is as follows: ； In the formula, Loss represents the loss function value of the first monitoring module, Q represents the total number of samples, represents the true label of the qth sample, represents the predicted value of the first monitoring module for the qth sample; Step S37: constructing a module and training it, specifically, constructing a first monitoring module through the graph structure data construction, the enhanced graph attention layer construction, the gated time convolution layer construction, the establishment of residual connections and jump connections, the calculation model output and the design of the loss function, and performing module training based on the sensor training set, verifying the module performance based on the sensor test set, and obtaining the first monitoring module.

3. The bridge structure safety monitoring method based on artificial intelligence according to claim 1 is characterized in that: In step S4, the second monitoring module is constructed to construct a module for processing bridge image data, and the second monitoring module specifically includes a residual network submodule, a visual transformer submodule, a feature fusion submodule and an output submodule; The second monitoring module is constructed, specifically comprising the following steps: Step S41: feature extraction, specifically using the residual network submodule and the visual transformer submodule to perform feature extraction Step S42: constructing a feature fusion submodule, the steps include: Step S421: Preliminary feature processing, the formula used is as follows: ； In the formula, represents the flattening function, represents the GELU activation function, represents the batch normalization operation function, represents the convolution operation function, represents the features extracted by the residual network submodule, represents the features extracted by the visual transformer submodule, represents the features extracted by the residual network submodule after preliminary processing, represents the features extracted by the visual transformer submodule after preliminary processing; Step S422: Design a self-attention mechanism. The formula used is as follows: ； Where Qu represents the query vector, Ke represents the key vector, and Va represents the value vector. represents the weight matrix used to calculate the query vector, represents the weight matrix used to calculate the key vector, represents the weight matrix used to calculate the value vector, represents the layer normalization operation function, S represents the attention score, T represents the transposition operation, represents the dimension of the key vector; Step S423: Calculate the submodule output using the following formula: ； In the formula, Os represents the output of the feature fusion submodule, represents the weight matrix used to control the linear transformation of the attention score, represents the bias term used to control the linear transformation of the attention score, represents the pooling operation function, Represents element-by-element addition operation; Step S424: submodule construction, specifically, constructing a feature fusion submodule through the feature preliminary processing, the designed self-attention mechanism and the calculation submodule output, to obtain the feature fusion submodule; Step S43: Calculate module output, the formula used is as follows: ； In the formula, Indicates the output of the second monitoring module, represents the output weight of the second monitoring module output submodule, represents the output bias item of the output submodule of the second monitoring module; Step S44: designing a loss function, the steps include: Step S441: Calculate the attention matrix. The formula used is as follows: ； Where Am represents the attention matrix; Step S442: Calculate the cross-transposed attention matrix, using the following formula: ； Where Ca represents the cross-transposed attention matrix; Step S443: Calculate the total loss using the following formula: ； In the formula, represents the cross entropy loss value of the second monitoring module, represents the cross entropy loss value based on the cross-transposed attention matrix of the second monitoring module, N represents the total number of image samples, represents the true label of the nth image sample, represents the predicted value of the second monitoring module for the nth image sample, M represents the total number of image sample categories, represents the weight of the mth category, Ta represents the attention matrix generated based on the true label, represents the element of the nth row and mth column of the attention matrix generated based on the true label, represents the element of the cross-transposed attention matrix in the nth row and mth column, Indicates the total loss value of the second monitoring module; Step S45: constructing a module and training it, specifically, constructing a second monitoring module through the feature extraction, constructing a feature fusion submodule, the calculation module output and the design loss function, and performing module training based on the image training set, and verifying the module performance based on the image test set to obtain a second monitoring module.

4. The bridge structure safety monitoring method based on artificial intelligence according to claim 1 is characterized in that: In step S5, the bridge safety monitoring model is constructed, specifically, a bridge safety monitoring model is constructed. The bridge safety monitoring model specifically includes a first monitoring module, a second monitoring module and a comprehensive output module. The comprehensive output module is used to integrate the output of the first monitoring module and the output of the second monitoring module, and output the prediction result of the model, specifically, by performing weighted summation on the output of the first monitoring module and the output of the second monitoring module to obtain comprehensive bridge safety reference data.

5. The bridge structure safety monitoring method based on artificial intelligence according to claim 1 is characterized in that: In step S1, the data collection is used to collect the original data required for monitoring the safety of the bridge structure, specifically, to obtain the original data set of the bridge condition through data collection; The original data set of bridge conditions specifically includes an original image data set and an original sensor data set. The bridge image data specifically includes an image of the overall structure of the bridge, a close-up image of a part of the bridge, an image of a bridge connection node, and a real-time monitoring image of the bridge. The bridge sensor data specifically includes bridge displacement data, bridge strain gauge data, bridge acceleration data, bridge temperature data, and bridge pressure data.

6. The bridge structure safety monitoring method based on artificial intelligence according to claim 1 is characterized by: In step S2, the data preprocessing is used to preprocess the collected raw data, and specifically includes the following steps: Step S21: sensor data preprocessing, specifically, performing data cleaning, data labeling, data encoding, data normalization and data set segmentation on the original sensor data set to obtain a sensor training set and a sensor test set; Step S22: image data preprocessing, specifically performing image cleaning, image expansion, image annotation and data set segmentation on the original image data set to obtain an image training set and an image test set; Step S23: preprocessing, specifically obtaining a sensor training set and a sensor test set by preprocessing the sensor data, and obtaining an image training set and an image test set by preprocessing the image data.

7. The bridge structure safety monitoring method based on artificial intelligence according to claim 1 is characterized by: In step S6, the bridge structure safety monitoring is specifically to analyze the safety status of the bridge structure based on the comprehensive bridge safety reference data, thereby realizing the monitoring of the bridge structure safety.

8. An artificial intelligence-based bridge structure safety monitoring system, used to implement the artificial intelligence-based bridge structure safety monitoring method as described in any one of claims 1 to 7, characterized in that: It includes a data acquisition module, a data preprocessing module, a first monitoring module construction module, a second monitoring module construction module, a bridge safety monitoring model construction module and a bridge structure safety monitoring module.

9. The artificial intelligence-based bridge structure safety monitoring system according to claim 8 is characterized in that: The data acquisition module is used for data acquisition, and obtains an original data set of bridge conditions through data acquisition, and sends the original data set of bridge conditions to the data preprocessing module; The data preprocessing module is used for data preprocessing, and obtains a sensor training set, a sensor test set, an image training set and an image test set through data preprocessing, and sends the sensor training set and the sensor test set to the first monitoring module construction module, and sends the image training set and the image test set to the second monitoring module construction module; The first monitoring module construction module is used to construct the first monitoring module, obtain the first monitoring module by constructing the first monitoring module, and send the first monitoring module to the bridge safety monitoring model construction module; The second monitoring module construction module is used to construct the second monitoring module, obtain the second monitoring module by constructing the second monitoring module, and send the second monitoring module to the bridge safety monitoring model construction module; The bridge safety monitoring model construction module is used to construct a bridge safety monitoring model, obtain comprehensive bridge safety reference data by constructing the bridge safety monitoring model, and send the comprehensive bridge safety reference data to the bridge structure safety monitoring module; The bridge structure safety monitoring module is used for bridge structure safety monitoring, and realizes monitoring of bridge structure safety by analyzing the safety status of the bridge structure based on the comprehensive bridge safety reference data.

Images