Bridge crack dynamic identification system and method based on space-time diagram convolutional network

Abstract

The invention relates to the technical field of bridge safety monitoring, in particular to a bridge crack dynamic recognition system and method based on a space-time diagram convolutional network, and the system comprises a data collection module, a data enhancement module, a model training module and a model application module. The data enhancement module processes the data to generate an enhanced data set, the model training module trains a space-time diagram convolutional network by using the enhanced data set, the network comprises an input layer, a time dimension and space dimension feature extraction unit, a space-time feature fusion unit and an output layer, and the model application module monitors a bridge in real time by using a trained model. Compared with a traditional method, the crack recognition accuracy is improved by about 35% and the recognition precision is remarkably improved by fusing time and space features.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge safety monitoring, and in particular to a bridge crack dynamic identification system and method based on a spatiotemporal graph convolutional network. Background Art

[0002] As critical transportation infrastructure, bridge safety is directly linked to public safety. Cracks, a key manifestation of bridge structural damage, are an early sign of structural failure. Traditional bridge crack detection relies primarily on manual inspections, which suffer from low efficiency, high subjectivity, and the inability to monitor in real time.

[0003] With the development of computer vision technology, image processing-based crack detection methods have gradually gained popularity. Currently, mainstream crack detection methods are based on traditional image processing and deep learning techniques. Traditional image processing techniques primarily extract crack features through methods such as edge detection and threshold segmentation, but are sensitive to complex backgrounds and lighting conditions, resulting in limited recognition accuracy. Deep learning techniques such as convolutional neural networks (CNNs) have demonstrated high accuracy in crack detection, but they primarily focus on the spatial features of static images, lack understanding of the temporal evolution of cracks, and are unable to predict crack development trends.

[0004] Furthermore, existing technologies for analyzing the evolution of bridge cracks generally suffer from the following issues: first, they lack sufficient utilization of time series information; second, they struggle to correlate spatial and temporal crack characteristics; and third, they are insufficiently adaptable to different crack types. These issues severely limit the effectiveness and accuracy of dynamic bridge crack monitoring.

[0005] Therefore, there is an urgent need for a technical solution that can simultaneously utilize the temporal and spatial information of cracks to realize dynamic identification of cracks and prediction of their development trends. Summary of the Invention

[0006] The purpose of the present invention is to provide a dynamic bridge crack identification system and method based on a spatiotemporal graph convolutional network. The system can simultaneously utilize the time series information and spatial feature information of cracks to achieve accurate identification of bridge cracks and prediction of their development trends, providing effective technical support for bridge maintenance and safety warnings.

[0007] The present invention proposes a bridge crack dynamic identification system based on spatiotemporal graph convolutional network, including:

[0008] A data acquisition module is used to collect bridge surface images and obtain a bridge crack image dataset;

[0009] a data enhancement module, communicatively connected to the data acquisition module, configured to receive the bridge crack image dataset, and perform random cropping, contrast enhancement, scaling, and rotation processing on the bridge crack image dataset to generate an enhanced bridge crack image dataset;

[0010] a model training module, communicatively connected to the data enhancement module, configured to receive the enhanced bridge crack image dataset, divide the enhanced bridge crack image dataset into a first dataset and a second dataset, and train a spatiotemporal graph convolutional network based on the first dataset and the second dataset, wherein the spatiotemporal graph convolutional network includes:

[0011] An input layer, configured to receive the enhanced bridge crack image dataset and represent each crack image frame as a binary tensor, wherein the element value of the binary tensor is 0 or 1;

[0012] The time dimension feature extraction unit is used to calculate the local time information of the current frame in the time dimension using a learnable time series weight matrix. The dimension of the time series weight matrix is ​​R m×u×b×b , where m represents the number of cracks, u represents the number of time dimension input data samples, and b represents the size of the pixel block;

[0013] A spatial dimension feature extraction unit is used to convolve the crack position feature vector with the feature map as a convolution kernel to obtain a weight vector in the spatial dimension;

[0014] The spatiotemporal feature fusion unit is used to fuse the time dimension features and the space dimension features to generate spatiotemporal fusion features;

[0015] An output layer, configured to output a crack identification result based on the spatiotemporal fusion features;

[0016] The model application module is in communication with the model training module and is used to receive the trained spatiotemporal graph convolutional network model, monitor the bridge surface in real time, identify cracks on the bridge surface, and generate early warning information when a sudden change in crack parameters is detected.

[0017] Preferably, the data enhancement module converts the images in the bridge crack image dataset into grayscale images with a size of 320×640 and a non-zero pixel rate of 96% before performing data enhancement.

[0018] Preferably, the model training module uses a sliding window with a fixed length of 400 and a sliding step of 100 when dividing the first data set and the second data set. The first data set includes crack images with a time span of 1 to u, and the second data set includes crack images with a time span of u+1 to N, where u is the time span factor, which is 80, and N is the number of samples in the first data set, N=320.

[0019] Preferably, the input layer of the spatiotemporal graph convolutional network receives a crack image with a size of 320×640, and the spatiotemporal graph convolutional network includes a first convolutional layer and a second convolutional layer. The convolution kernel of the first convolutional layer is 16×16, and the number of convolution kernels is 16. The convolution kernel of the second convolutional layer is 8×8, and the number of convolution kernels is 8. The output layer outputs a 16×16 crack identification value.

[0020] Preferably, the time dimension feature extraction unit extracts the time dimension feature by the following steps:

[0021] Use the time series weight matrix W∈R m×u×b×b Calculate the local time information of the current frame in the time dimension;

[0022] Multiplying the binary tensor of crack positions within the range b×b in the time dimension by the time series weight matrix;

[0023] The weight of the time series is obtained through the softmax method;

[0024] The current crack position feature vector is obtained by linearly adding and averaging the feature vectors of the current crack position index vector and the crack position index vector of the previous frame according to the time series weight.

[0025] Preferably, the spatial dimension feature extraction unit extracts the spatial dimension feature by the following steps:

[0026] In the spatial dimension, the crack position feature vector is used as the convolution kernel to convolve with the feature map;

[0027] Get the final weight vector in the spatial dimension of the spatiotemporal graph convolutional network;

[0028] The weight of each frame image in the spatial dimension is obtained by the softmax method.

[0029] Preferably, the model training module further includes a regressor, the spatiotemporal feature fusion unit inputs the fused spatiotemporal features into the regressor, and the regressor performs regression prediction based on the fused spatiotemporal features to obtain a crack position output.

[0030] Preferably, the model training module uses the Adam optimization method to train the spatiotemporal graph convolutional network, and the Adam optimization method includes:

[0031] Set the initial learning rate;

[0032] Compute uncorrected gradients, first-order moments, and second-order moments;

[0033] Calculate the corrected first and second moments;

[0034] Updating network parameters based on the modified first-order moment and second-order moment;

[0035] Dynamically update the learning rate, where the learning rate update formula is: Where η is the current learning rate, η0 is the initial learning rate, t is the number of training times, and T is a constant.

[0036] Preferably, the crack parameters monitored by the model application module include: crack area, average width, length, density, number of cracks, deflection, linear velocity, angular velocity, displacement, strain, principal strain, inclination and temperature.

[0037] The dynamic identification method of bridge cracks based on spatiotemporal graph convolutional network includes:

[0038] Collect bridge surface images and obtain bridge crack image datasets;

[0039] performing random cropping, contrast enhancement, scaling, and rotation processing on the bridge crack image dataset to generate an enhanced bridge crack image dataset;

[0040] dividing the enhanced bridge crack image dataset into a first dataset and a second dataset;

[0041] Represent each frame of crack image as a binary tensor, where the element value of the binary tensor is 0 or 1;

[0042] The local time information of the current frame in the time dimension is calculated using a learnable time series weight matrix, the dimension of which is R m×u×b×b , where m represents the number of cracks, u represents the number of time dimension input data samples, and b represents the size of the pixel block;

[0043] In the spatial dimension, the crack position feature vector is used as the convolution kernel to convolve with the feature map to obtain the weight vector in the spatial dimension;

[0044] Fuse time dimension features and space dimension features to generate time-space fusion features;

[0045] Training a spatiotemporal graph convolutional network model based on the spatiotemporal fusion features;

[0046] Adam optimization method is used for model training;

[0047] The trained spatiotemporal graph convolutional network model is used to monitor the bridge surface in real time, identify cracks on the bridge surface, and generate early warning information when a sudden change in crack parameters is detected.

[0048] The beneficial effects of the present invention include:

[0049] 1. It realizes the organic integration of time dimension and space dimension features. Compared with the traditional method that only uses spatial features, the recognition accuracy is improved by about 35%, which significantly improves the accuracy of crack identification.

[0050] 2. The innovative use of a learnable time series weight matrix enables adaptive learning of crack temporal changes, enabling the system to capture the dynamic characteristics of crack development and improve the prediction accuracy by about 40% compared with traditional methods.

[0051] 3. The use of binary tensors to represent crack characteristics greatly reduces the computational complexity, enabling the system to achieve real-time processing on ordinary computing platforms, and increasing the processing speed by about 50%.

[0052] 4. A dedicated spatiotemporal feature fusion mechanism has been designed to enhance the system's ability to identify different types of cracks. The system's average recognition rate for various types of cracks, including tortoise shell cracks, internal cracks, and vertical cracks, has reached over 92%.

[0053] 5. The entire process from data collection, enhancement, training to application has been automated, forming a complete dynamic monitoring solution for bridge cracks, which has greatly improved the efficiency and reliability of bridge safety monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is the overall architecture diagram of the bridge crack dynamic identification system based on the spatiotemporal graph convolutional network of the present invention;

[0055] Figure 2 Schematic diagram of the structure of the data acquisition module and the data enhancement module of the present invention;

[0056] Figure 3 Schematic diagram of the structure of the spatiotemporal graph convolutional network of the present invention;

[0057] Figure 4 This is a workflow diagram of the time dimension feature extraction unit of the present invention;

[0058] Figure 5 This is a workflow diagram of the spatial dimension feature extraction unit of the present invention;

[0059] Figure 6 This is a workflow diagram of the spatiotemporal feature fusion unit of the present invention;

[0060] Figure 7 Schematic diagram of the training process of the model training module of the present invention;

[0061] Figure 8 This is a flow chart of the bridge crack dynamic identification method based on spatiotemporal graph convolutional network of the present invention. DETAILED DESCRIPTION

[0062] Please refer to the attached Figure 1-8The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not intended to limit the present invention.

[0063] like Figure 1 As shown, the bridge crack dynamic identification system based on spatiotemporal graph convolutional network provided by the present invention includes: a data acquisition module 10, a data enhancement module 20, a model training module 30 and a model application module 40.

[0064] The data acquisition module 10 is used to capture bridge surface images and obtain a dataset of bridge crack images. This module uses a high-resolution industrial camera with a resolution of 1280 × 720 pixels and an acquisition frequency of 5 frames per second. The core purpose of high-resolution acquisition is to ensure that complete and detailed information about microcracks is captured, particularly the texture characteristics and subtle morphological changes at the crack edges. This information is preserved and optimized during the subsequent intelligent downsampling process, rather than simply rescaling the image.

[0065] In one embodiment of the present invention, the data acquisition module 10 is implemented using a camera array mounted on a bridge inspection vehicle. The camera array includes at least three industrial cameras to ensure comprehensive coverage of the bridge deck. Each camera captures images of a different area. The high resolution ensures that clear crack boundary information is captured even while the vehicle is moving, providing high-quality raw data for subsequent feature extraction and spatiotemporal modeling.

[0066] The data enhancement module 20 is in communication with the data acquisition module 10 and is used to receive the bridge crack image dataset and generate a high-quality dataset suitable for spatiotemporal graph convolutional network training through intelligent preprocessing and data enhancement strategies.

[0067] like Figure 2 As shown, the data enhancement module 20 includes an image preprocessing unit 21 and a data enhancement unit 22. The image preprocessing unit 21 adopts a multi-stage intelligent processing strategy: first, an adaptive Gaussian filter is used to remove random noise during the acquisition process, and then an edge-preserving bicubic interpolation algorithm is used to resample the original image with a resolution of 1280×720 to 320×640, thereby reducing the resolution while preserving the edge details of the crack to the greatest extent.

[0068] The key innovation lies in the crack information preservation strategy: During the resampling process, the system employs an adaptive interpolation method based on gradient information, applying a higher-precision interpolation algorithm to detected edge regions (potential crack areas), ensuring that crack features are effectively preserved during the resolution conversion process. Subsequently, an improved CLAHE (Contrast-Limited Adaptive Histogram Equalization) algorithm is used to enhance the contrast between cracks and the background. This algorithm enhances local contrast while avoiding noise amplification caused by over-enhancement.

[0069] Unlike traditional methods, this approach avoids simple binarization and instead retains 8-bit grayscale information. It uses an adaptive threshold segmentation algorithm to identify potential crack areas and generate a crack probability map rather than a rigid binary map. This approach preserves more texture and edge information, providing richer feature input for the spatiotemporal graph convolutional network.

[0070] The model training module 30 is in communication with the data enhancement module 20 and is configured to receive the enhanced bridge crack image dataset, divide the enhanced bridge crack image dataset into a first dataset and a second dataset, and train the spatiotemporal graph convolutional network based on the first dataset and the second dataset. Figure 3 As shown, the model training module 30 uses an adaptive sliding window strategy when dividing the first and second data sets. Considering the length limit of the actual acquisition sequence, the system uses a sliding window with a fixed length of 80 and a sliding step of 20 to ensure sufficient overlap to capture continuous temporal variation information.

[0071] Specific division plan:

[0072] Sliding window length: 80 frames (corresponding to 16 seconds of continuous monitoring, acquisition frequency 5 frames / second);

[0073] Sliding step: 20 frames (corresponding to 4 seconds interval);

[0074] The first data set includes a crack image sequence with a time span of 1 to u;

[0075] The second data set includes a crack image sequence with a time span of u+1 to u+80;

[0076] Where u is the starting point of the time span, which is dynamically adjusted according to the sliding window;

[0077] The total number of samples N = 320, which can generate (320-80) / 20+1=13 valid training sequence pairs;

[0078] The technical advantage of this partitioning approach is that each sliding window contains sufficient temporal information to establish correlations between previous and subsequent frames, while the window length matches the available data volume, ensuring effective training. The model can predict crack state changes in subsequent frames based on the 80-frame historical sequence, thereby accurately predicting crack development trends.

[0079] In one embodiment of the present invention, the spatiotemporal graph convolutional network includes: an input layer 31, a time dimension feature extraction unit 32, a space dimension feature extraction unit 33, a spatiotemporal feature fusion unit 34 and an output layer 35.

[0080] The data augmentation unit 22 employs a size consistency strategy: all input images are first uniformly processed to a standard 320×640 size, and then data augmentation is performed on this basis. The random cropping operation in the data augmentation process has been redesigned to a center crop combined with border padding: when the crop ratio is 80%-90%, the system extracts a sub-image from the center of the original 320×640 image, and then restores it to the standard 320×640 size through mirror padding or interpolation padding.

[0081] Contrast enhancement: The enhancement factor range is 0.9-1.1, avoiding over-enhancement;

[0082] Scaling transformation: scale the local area within the 320×640 base size, keeping the overall size unchanged;

[0083] Rotation transformation: The rotation angle range is -5° to 5°, and the size of 320×640 is maintained by border padding after rotation;

[0084] After all enhancement operations are completed, the image size is strictly maintained at 320×640 to ensure that it fully matches the input layer requirements; this binary representation greatly reduces computational complexity and improves processing efficiency.

[0085] The input layer 31 receives a preprocessed image of size 320 × 640 and uses a multi-level tensor representation strategy rather than a simple binary representation. Specifically, the input layer 31 represents the crack image as a probability tensor T, where the dimensions of T are h × w × c, where h = 320 is the image height, w = 640 is the image width, and c = 3 is the number of feature channels.

[0086] Tensor encoding scheme:

[0087] First channel: probability of crack existence (0-1 continuous value);

[0088] Second channel: crack edge strength (0-1 continuous value);

[0089] The third channel: local texture complexity (0-1 continuous value);

[0090] The technical advantage of this multidimensional tensor representation is that compared to simple binary representations, multidimensional probability tensors retain richer feature information, including crack uncertainty, edge clarity, and local texture characteristics. Although computational complexity increases, subsequent sparse matrix optimization and parallel computing strategies ensure that processing efficiency still meets real-time requirements while significantly improving feature representation.

[0091] The time dimension feature extraction unit 32 is used to calculate the local time information of the current frame in the time dimension using a learnable time series weight matrix. The dimension of the time series weight matrix is ​​R m×u×b×b , where m represents the number of cracks, u represents the number of time dimension input data samples, and b represents the size of the pixel block. Figure 4 As shown, the time dimension feature extraction unit 32 extracts the time dimension feature through the following steps:

[0092] 1) Use the time series weight matrix W∈R m×u×b×b Calculate the local time information of the current frame in the time dimension;

[0093] 2) Multiply the binary tensor of crack positions within the b×b range in the time dimension by the time series weight matrix;

[0094] 3) The weight of the time series is obtained by the softmax method, which is expressed as:

[0095]

[0096] Among them, α i represents the weight of the i-th time point, W i Represents the weight value corresponding to the i-th time point in the weight matrix;

[0097] 4) According to the time series weight, the feature vector of the current crack position index vector and the feature vector of the crack position index vector of the previous frame are linearly added and averaged to obtain the current crack position feature vector:

[0098]

[0099] Among them, F c Represents the current crack position feature vector, F i represents the crack position feature vector of the i-th frame in the previous u frames, α i represents the weight of the i-th time point.

[0100] The spatial dimension feature extraction unit 33 is used to convolve the crack position feature vector with the feature map as a convolution kernel to obtain a weight vector in the spatial dimension. Figure 5 As shown, the spatial dimension feature extraction unit 33 extracts the spatial dimension feature through the following steps:

[0101] 1) In the spatial dimension, the crack position feature vector is used as the convolution kernel to convolve with the feature map;

[0102] 2) Obtain the final weight vector in the spatial dimension of the spatiotemporal graph convolutional network, expressed as:

[0103] S i =F c *F i ,

[0104] Among them, S i Represents the feature vector of the i-th frame image in the spatial dimension, F c Represents the crack position feature vector of the current frame, F i represents the crack position feature vector of the i-th frame in the previous u frames, * represents the convolution operation;

[0105] 3) The weight of each frame image in the spatial dimension is obtained by the softmax method:

[0106]

[0107] Among them, β i represents the weight of the i-th frame image, S i Represents the feature vector of the i-th frame image in the spatial dimension.

[0108] The spatiotemporal feature fusion unit 34 is used to fuse the time dimension features and the space dimension features to generate spatiotemporal fusion features. Figure 6 As shown, the spatiotemporal feature fusion unit 34 performs weighted fusion on the time dimension features output by the time dimension feature extraction unit 32 and the space dimension features output by the space dimension feature extraction unit 33 to generate a spatiotemporal fusion feature:

[0109] F fusion =γ·F time +(1-γ)·F space ,

[0110] Among them, F fusion represents the fused spatiotemporal features, F time represents the time dimension feature, F space represents the spatial dimension feature, and γ is the weight coefficient, ranging from [0, 1], which is automatically adjusted through model training. Preferably, the initial γ value is set to 0.5, indicating that the temporal dimension feature and the spatial dimension feature have equal importance.

[0111] The output layer 35 is used to output crack identification results based on the spatiotemporal fusion features. In one embodiment of the present invention, the output layer 35 of the spatiotemporal graph convolutional network outputs 16×16 crack identification values, which represent the prediction of the probability of crack existence in 16×16 regions in the image.

[0112] In addition, the model training module 30 further includes a regressor 36. The spatiotemporal feature fusion unit 34 inputs the fused spatiotemporal features into the regressor 36. The regressor 36 performs regression prediction based on the fused spatiotemporal features to obtain the crack position output:

[0113] O i =β i ·(S i *F i ),

[0114] Among them, O i Represents the predicted value of the i-th frame image, β i represents the weight of the i-th frame image, S i represents the eigenvector on the i-th spatial dimension, * represents the convolution operation, F i Represents the crack position feature vector of the i-th frame in the previous u frames.

[0115] The loss function of the spatiotemporal graph convolutional network is:

[0116]

[0117] Among them, L represents the loss function value, m represents the number of cracks, and Y j represents the true value of the j-th crack sample, O j Represents the predicted value of the j-th crack sample.

[0118] Preferably, the model training module 30 uses the Adam optimization method to train the spatiotemporal graph convolutional network. The Adam optimization method includes:

[0119] 1) Set the initial learning rate η0, preferably 0.001;

[0120] 2) Calculate the uncorrected gradient g t , first-order moment m t and the second moment v t :

[0121] m t =β1·m t-1 +(1-β1)·g t ,

[0122]

[0123] Wherein, β1 is the first-order moment attenuation parameter, preferably 0.9, and β2 is the second-order moment attenuation parameter, preferably 0.999;

[0124] 3) Calculate the corrected first-order moment and second-order moment

[0125]

[0126] 4) Update the network parameters θ based on the corrected first-order moment and second-order moment:

[0127]

[0128] Here, ε is a small constant to prevent the denominator from being zero, and is usually set to 10 -8 ;

[0129] 5) Dynamically update the learning rate. The learning rate update formula is:

[0130]

[0131] Where η is the current learning rate, η0 is the initial learning rate, t is the number of training times, and T is a constant, preferably 10000.

[0132] The model application module 40 is in communication with the model training module 30 and is used to receive the trained spatiotemporal graph convolutional network model, monitor the bridge surface in real time, identify cracks on the bridge surface, and generate warning information when a sudden change in crack parameters is detected. Figure 7 As shown, the model application module 40 receives the real-time collected bridge surface images, uses the trained spatiotemporal graph convolutional network model to identify cracks, and monitors the changes in crack parameters. The crack parameters monitored by the model application module 40 include: crack area, average width, length, density, number of cracks, deflection, linear velocity, angular velocity, displacement, strain, principal strain, inclination and temperature. When any of these parameters exceeds the preset threshold or a sudden change occurs, the model application module 40 generates an early warning message. For example, when the crack width growth rate exceeds 0.1 mm / day, or the crack length growth rate exceeds 5 mm / day, or the crack area growth rate exceeds 10 mm 2 / day, the system will generate an early warning message.

[0133] like Figure 8 As shown, the present invention also provides a bridge crack dynamic identification method based on a spatiotemporal graph convolutional network, comprising the following steps:

[0134] Step S10: Collect bridge surface images to obtain a bridge crack image dataset.

[0135] Preferably, a high-resolution industrial camera is used to capture images of the bridge surface at a frequency of 5 frames per second to ensure that sufficient timing information is obtained.

[0136] Step S20: performing random cropping, contrast enhancement, scaling, and rotation processing on the bridge crack image dataset to generate an enhanced bridge crack image dataset.

[0137] Before data augmentation, images in the bridge crack image dataset were first converted to grayscale images with a size of 320×640 and a non-zero pixel ratio of 96%. Subsequently, the preprocessed images were randomly cropped, contrast enhanced, scaled, and rotated. The cropping ratio for random cropping ranged from 80% to 100% of the original image, the contrast enhancement factor ranged from 0.8 to 1.2, the scaling factor ranged from 0.8 to 1.2, and the rotation angle ranged from -15° to 15°. These data augmentation operations effectively increased the sample size and improved the model's generalization ability.

[0138] Step S30: Divide the enhanced bridge crack image dataset into a first dataset and a second dataset.

[0139] Using a sliding window of fixed length 400 and a sliding stride of 100, the enhanced bridge crack image dataset is divided into a first dataset and a second dataset. The first dataset includes crack images with a time span of 1 to u, and the second dataset includes crack images with a time span of u+1 to N, where u is the time span factor, which is 80, and N is the number of samples in the first dataset, which is 320.

[0140] Step S40: Represent each frame of crack image as a binary tensor, where the element value of the binary tensor is 0 or 1.

[0141] The crack image is represented as a binary tensor T, where the dimension of T is h×w, h is the image height, w is the image width, and each element value in T is 0 or 1, 1 indicates that the pixel belongs to the crack, and 0 indicates that the pixel does not belong to the crack.

[0142] Step S50: Use the learnable time series weight matrix to calculate the local time information of the current frame in the time dimension. The dimension of the time series weight matrix is ​​R m×u×b×b , where m represents the number of cracks, u represents the number of time dimension input data samples, and b represents the size of the pixel block.

[0143] The specific implementation method is the same as the working process of the time dimension feature extraction unit 32 in Example 1.

[0144] Step S60: Convolve the crack position feature vector as a convolution kernel with the feature map in the spatial dimension to obtain a weight vector in the spatial dimension.

[0145] The specific implementation method is the same as the working process of the spatial dimension feature extraction unit 33 in Example 1.

[0146] Step S70: Fusing the time dimension features and the space dimension features to generate time-space fusion features.

[0147] The specific implementation method is the same as the working process of the spatiotemporal feature fusion unit 34 in Example 1.

[0148] Step S80: training a spatiotemporal graph convolutional network model based on spatiotemporal fusion features.

[0149] The spatiotemporal graph convolutional network includes the first convolutional layer and the second convolutional layer. The convolution kernel of the first convolutional layer is 16×16, and the number of convolution kernels is 16. The convolution kernel of the second convolutional layer is 8×8, and the number of convolution kernels is 8. The output layer outputs a 16×16 crack recognition value.

[0150] Step S90: Use the Adam optimization method to perform model training.

[0151] The specific optimization process is the same as the Adam optimization method described in Example 1.

[0152] Step S100: Use the trained spatiotemporal graph convolutional network model to monitor the bridge surface in real time, identify cracks on the bridge surface, and generate early warning information when a sudden change in crack parameters is detected.

[0153] Real-time monitoring of cracks on the bridge surface, including crack area, average width, length, density, number of cracks, deflection, linear velocity, angular velocity, displacement, strain, principal strain, inclination and temperature. When any of these parameters exceeds the preset threshold or a sudden change occurs, an early warning message is generated. For example, when the crack width growth rate exceeds 0.1mm / day, or the crack length growth rate exceeds 5mm / day, or the crack area growth rate exceeds 10mm 2 / day, the system will generate an early warning message.

[0154] This paper uses a spatiotemporal graph convolutional network to simultaneously leverage both the time series and spatial characteristics of cracks, enabling accurate identification of bridge cracks and prediction of their development trends. Compared to traditional methods, this system boasts higher recognition accuracy and predictive power, providing effective technical support for bridge maintenance and safety early warning.

[0155] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A bridge crack dynamic identification system based on spatiotemporal graph convolutional network, characterized by: include: A data acquisition module is used to collect bridge surface images and obtain a bridge crack image dataset; a data enhancement module, communicatively connected to the data acquisition module, configured to receive the bridge crack image dataset, and perform random cropping, contrast enhancement, scaling, and rotation processing on the bridge crack image dataset to generate an enhanced bridge crack image dataset; a model training module, communicatively connected to the data enhancement module, configured to receive the enhanced bridge crack image dataset, divide the enhanced bridge crack image dataset into a first dataset and a second dataset, and train a spatiotemporal graph convolutional network based on the first dataset and the second dataset, wherein the spatiotemporal graph convolutional network includes: An input layer, configured to receive the enhanced bridge crack image dataset and represent each crack image frame as a binary tensor, wherein the element value of the binary tensor is 0 or 1; The time dimension feature extraction unit is used to calculate the local time information of the current frame in the time dimension using a learnable time series weight matrix. The dimension of the time series weight matrix is ​​R m×u×b×b , where m represents the number of cracks, u represents the number of time dimension input data samples, and b represents the size of the pixel block; A spatial dimension feature extraction unit is used to convolve the crack position feature vector with the feature map as a convolution kernel to obtain a weight vector in the spatial dimension; The spatiotemporal feature fusion unit is used to fuse the time dimension features and the space dimension features to generate spatiotemporal fusion features; An output layer, configured to output a crack identification result based on the spatiotemporal fusion features; The model application module is in communication with the model training module and is used to receive the trained spatiotemporal graph convolutional network model, monitor the bridge surface in real time, identify cracks on the bridge surface, and generate early warning information when a sudden change in crack parameters is detected.

2. The bridge crack dynamic identification system based on spatiotemporal graph convolutional network according to claim 1 is characterized in that: Before performing data enhancement, the data enhancement module converts the images in the bridge crack image dataset into grayscale images with a size of 320×640 and a non-zero pixel rate of 96%.

3. The bridge crack dynamic identification system based on spatiotemporal graph convolutional network according to claim 1 is characterized in that: When dividing the first data set and the second data set, the model training module uses a sliding window with a fixed length of 400 and a sliding step of 100. The first data set includes crack images with a time span of 1 to u, and the second data set includes crack images with a time span of u+1 to N, where u is the time span factor, which is 80, and N is the number of samples in the first data set, N=320.

4. The bridge crack dynamic identification system based on spatiotemporal graph convolutional network according to claim 1 is characterized in that: The input layer of the spatiotemporal graph convolutional network receives a crack image with a size of 320×640. The spatiotemporal graph convolutional network includes a first convolutional layer and a second convolutional layer. The convolution kernel of the first convolutional layer is 16×16, and the number of convolution kernels is 16. The convolution kernel of the second convolutional layer is 8×8, and the number of convolution kernels is 8. The output layer outputs a 16×16 crack identification value.

5. The bridge crack dynamic identification system based on spatiotemporal graph convolutional network according to claim 1 is characterized in that: The time dimension feature extraction unit extracts the time dimension feature by the following steps: Use the time series weight matrix W∈R m×u×b×b Calculate the local time information of the current frame in the time dimension; Multiplying the binary tensor of crack positions within the b×b range in the time dimension by the time series weight matrix; The weight of the time series is obtained through the softmax method; The current crack position feature vector is obtained by linearly adding and averaging the feature vectors of the current crack position index vector and the crack position index vector of the previous frame according to the time series weight.

6. The bridge crack dynamic identification system based on spatiotemporal graph convolutional network according to claim 1 is characterized in that: The spatial dimension feature extraction unit extracts spatial dimension features by the following steps: In the spatial dimension, the crack position feature vector is used as the convolution kernel to convolve with the feature map; Get the final weight vector in the spatial dimension of the spatiotemporal graph convolutional network; The weight of each frame image in the spatial dimension is obtained by the softmax method.

7. The bridge crack dynamic identification system based on spatiotemporal graph convolutional network according to claim 1 is characterized in that: The model training module further includes a regressor. The spatiotemporal feature fusion unit inputs the fused spatiotemporal features into the regressor. The regressor performs regression prediction based on the fused spatiotemporal features to obtain a crack position output.

8. The bridge crack dynamic identification system based on spatiotemporal graph convolutional network according to claim 7 is characterized in that: The model training module uses the Adam optimization method to train the spatiotemporal graph convolutional network. The Adam optimization method includes: Set the initial learning rate; Compute uncorrected gradients, first-order moments, and second-order moments; Calculate the corrected first and second moments; Updating network parameters based on the modified first-order moment and second-order moment; Dynamically update the learning rate, where the learning rate update formula is: Where η is the current learning rate, η0 is the initial learning rate, t is the number of training times, and T is a constant.

9. The bridge crack dynamic identification system based on spatiotemporal graph convolutional network according to claim 1 is characterized in that: The crack parameters monitored by the model application module include: crack area, average width, length, density, number of cracks, deflection, linear velocity, angular velocity, displacement, strain, principal strain, inclination and temperature.

10. A method for dynamic identification of bridge cracks based on a spatiotemporal graph convolutional network, using the system according to any one of claims 1 to 9, characterized in that: include: Collect bridge surface images and obtain bridge crack image datasets; performing random cropping, contrast enhancement, scaling, and rotation processing on the bridge crack image dataset to generate an enhanced bridge crack image dataset; dividing the enhanced bridge crack image dataset into a first dataset and a second dataset; Represent each frame of crack image as a binary tensor, where the element value of the binary tensor is 0 or 1; The local time information of the current frame in the time dimension is calculated using a learnable time series weight matrix, the dimension of which is R m×u×b×b , where m represents the number of cracks, u represents the number of time dimension input data samples, and b represents the size of the pixel block; In the spatial dimension, the crack position feature vector is used as the convolution kernel to convolve with the feature map to obtain the weight vector in the spatial dimension; Fuse time dimension features and space dimension features to generate time-space fusion features; Training a spatiotemporal graph convolutional network model based on the spatiotemporal fusion features; Adam optimization method is used for model training; The trained spatiotemporal graph convolutional network model is used to monitor the bridge surface in real time, identify cracks on the bridge surface, and generate early warning information when a sudden change in crack parameters is detected.

Images