A concrete crack development trend prediction method and system based on time sequence images

Abstract

The application discloses a concrete crack development trend prediction method and system based on time sequence images, and the method comprises the following steps: acquiring time sequence image data sets of a concrete structure, calculating displacement vectors of crack edge pixel points of each image in the time sequence image data sets based on an optical flow method, and generating a crack propagation rate data set; obtaining crack time-varying characteristics based on the crack propagation rate data set and a preset crack trend prediction model; and then combining environmental time sequence data and a multi-modal fusion prediction network to predict a prediction result of the development trend of the concrete crack, so that the prediction capability for the development trend of the concrete crack is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for predicting the development trend of concrete cracks based on time-series images. Background Technology

[0002] Concrete is a brittle material that is prone to cracking when subjected to external forces or internal stresses. Common external factors include temperature changes, load effects, and variations in ambient humidity, especially when concrete structures experience uneven stress, settlement, or rapid temperature changes, which can easily lead to crack formation. Concrete crack development trend prediction involves analyzing time-series image data of cracks on the surface of concrete structures and combining this with environmental factors to predict the crack's development trend over a future period. Concrete crack development trend prediction not only helps optimize maintenance and reinforcement plans but also effectively avoids sudden structural failures, ensuring the long-term safe operation of the project.

[0003] Existing methods for predicting the development of concrete cracks typically rely on assumptions and simplified models, neglecting the time-varying characteristics and complex nonlinear effects of crack behavior, resulting in poor prediction accuracy and applicability. Secondly, existing methods have significant limitations in processing multimodal data, failing to effectively integrate multidimensional information, leading to inaccurate predictions of the development of concrete cracks. Summary of the Invention

[0004] This invention provides a method and system for predicting the development trend of concrete cracks based on time-series images, so as to improve the accuracy of predicting the development trend of concrete cracks.

[0005] To address the aforementioned technical problems, this invention provides a method for predicting the development trend of concrete cracks based on time-series images, comprising:

[0006] A time-series image dataset of a concrete structure is obtained, and the displacement vector of the crack edge pixels in each image in the time-series image dataset is calculated based on the optical flow method to generate a crack propagation rate dataset.

[0007] Based on the crack propagation rate dataset and the preset crack trend prediction model, the time-varying characteristics of the crack are obtained; the crack trend prediction model is constructed based on the LSMT network.

[0008] Collect environmental time-series data of the concrete structure, determine the temperature center, humidity center and load center based on the environmental time-series data, and construct a geometric triangular structure based on the temperature center, humidity center and load center;

[0009] The center of gravity of the geometric triangular structure is taken as the influence center of the concrete structure, and the temperature influence area, the humidity influence area and the load influence area are calculated based on the distance from the influence center to each side of the geometric triangular structure;

[0010] The temperature weight, the humidity weight and the load weight are determined based on the temperature influence area, the humidity influence area and the load influence area respectively, and the target multi-modal fusion prediction network is obtained by optimizing a preset multi-modal fusion prediction network based on the temperature weight, the humidity weight and the load weight.

[0011] The environmental time series data is input into the target multi-modal fusion prediction network to obtain crack environmental features.

[0012] The crack time-varying features and the crack environmental features are spliced to obtain fusion features, and a prediction result of the development trend of the concrete crack is generated based on the fusion features combined with a deep neural network.

[0013] The application can comprehensively and dynamically track the evolution process of the crack by collecting high-definition images of the surface of the concrete structure and constructing a time-series dataset based on the time-series images. The displacement vector of the crack edge pixel point in the time-series image data is calculated by the optical flow method to generate a crack propagation rate dataset. This process can accurately quantify the crack propagation and provide basic data for further crack development trend prediction. It can automatically and continuously track the development of the crack, avoiding the errors and omissions of traditional manual detection, and improving the monitoring accuracy and efficiency. Moreover, by introducing a crack trend prediction model, the time-varying characteristics can be deeply mined, and the dynamic law in the crack evolution process can be captured to provide a prediction basis for the future trend of the crack. The time-varying characteristics of the crack not only reflect the current state of the crack, but also can predict the possible future trend of the crack through historical data analysis, providing decision support for long-term structure maintenance and reinforcement. Furthermore, the environmental time-series data of the concrete structure are collected and processed, a geometric triangle structure based on the temperature vertex, humidity vertex and load vertex is constructed, and the center of gravity of the geometric triangle structure is taken as the influence center of the concrete structure, effectively quantifying the comprehensive influence of different environmental factors on the crack development. The influence area of temperature, humidity and load is calculated based on the distance from the influence center to each side of the geometric triangle, thereby reflecting the influence intensity of each environmental factor on the crack development of the concrete structure. Specifically, the temperature influence area, humidity influence area and load influence area can assign a weight to each environmental factor, and these weights reflect the importance of different environmental factors in the development process of the concrete crack. Based on the three weights as the optimization target input into the multi-modal fusion prediction network, the prediction accuracy of the network is further improved. The introduction of the weight not only considers the independent influence of each environmental factor, but also comprehensively considers the relationship between them, so that the prediction network can more accurately capture the dynamic changes of different environmental factors on the development trend of the concrete crack. The model not only improves the prediction ability of the development trend of the concrete crack, but also enhances the adaptability and accuracy of the model to complex environmental conditions. On this basis, the crack time-varying characteristics and crack environmental characteristics are spliced to form a fusion feature based on the deep neural network, and the prediction result of the crack development trend is generated based on the fusion feature. It can learn and extract complex data relationships and improve the prediction accuracy.

[0014] Further, the time-series image dataset of the concrete structure is obtained, and the displacement vector of the crack edge pixel point of each image in the time-series image dataset is calculated based on the optical flow method to generate a crack propagation rate dataset, comprising:

[0015] The time-series image dataset of the concrete structure is obtained, and the time-series image dataset is constructed from the high-definition images of the surface of the concrete structure.

[0016] obtain a crack candidate region and a corresponding crack edge pixel set of each time sequence image based on a preset semantic segmentation algorithm and the time sequence image dataset;

[0017] select feature points of each time sequence image from the crack edge pixel set based on a preset selection strategy;

[0018] calculate a pixel-level displacement vector of each feature point in the crack candidate region based on the optical flow method for time sequence images of any adjacent time frame;

[0019] calculate a physical displacement in an adjacent time interval based on the pixel-level displacement vector, and generate a crack propagation rate dataset based on the physical displacement.

[0020] The present application can accurately extract the crack candidate region and its edge pixels of each time sequence image by calculating the displacement vector of the crack edge pixels in the time sequence image data using the optical flow method and further generating the crack propagation rate dataset, thereby providing high-quality input data for subsequent displacement vector calculation. By calculating the pixel-level displacement vector based on the optical flow method and converting it into a physical displacement, the present application can obtain the actual expansion of the crack in the adjacent time interval, avoiding errors caused by manual intervention and subjective judgment. Moreover, by accurately calculating the pixel-level displacement, the crack propagation rate can be accurately quantified, providing key data for crack development trend prediction, achieving efficient and accurate monitoring of concrete crack propagation, and providing a scientific basis for subsequent crack prediction and early warning.

[0021] Further, the obtaining of the crack candidate region and the corresponding crack edge pixel set of each time sequence image based on the preset semantic segmentation algorithm and the time sequence image dataset comprises:

[0022] detecting the time sequence image dataset based on a preset target detection algorithm to obtain the crack candidate region of each time sequence image;

[0023] extracting a crack binary mask of the crack candidate region of each time sequence image based on a preset semantic segmentation algorithm, and refining the crack binary mask to obtain the crack edge pixel set.

[0024] The present application extracts the crack candidate region in the time sequence image through the semantic segmentation algorithm and the target detection algorithm, and further obtains the crack edge pixel set. Through the preset target detection algorithm, the crack region in the image can be automatically identified to ensure accurate positioning of the crack region. Then, the binary mask of the crack region is extracted through the semantic segmentation algorithm, and the accurate crack edge pixel set is obtained through fine processing, solving the complexity and uncertainty in crack detection, avoiding errors caused by manual annotation, and improving automation and precision.

[0025] Further, the physical displacement in the adjacent time interval is calculated based on the pixel-level displacement vector, and a crack propagation rate dataset is generated based on the physical displacement, including:

[0026] The pixel-level displacement vector is preprocessed to obtain target displacement vector data;

[0027] A preset camera calibration scale is obtained to convert the target displacement vector data into physical displacement;

[0028] The local crack propagation rate is calculated based on the physical displacement in the adjacent time interval, and the time stamp corresponding to the local crack propagation rate is determined to generate a crack propagation rate dataset.

[0029] The present application can accurately calculate the physical displacement in the adjacent time interval by preprocessing the pixel-level displacement vector calculated by the optical flow method to obtain target displacement vector data, and converting it into physical displacement based on the camera calibration scale. Further, the crack propagation rate dataset is generated, which not only ensures the accuracy of the crack propagation rate, but also tracks the time evolution characteristics of the crack development through the record of the time stamp, further improving the accuracy of the prediction model.

[0030] Further, the environmental time series data of the concrete structure is collected, and the temperature center, humidity center and load center are determined based on the environmental time series data, and a geometric triangular structure is constructed based on the temperature center, humidity center and load center, including:

[0031] The environmental time series data of the concrete structure is collected, and the temperature data, humidity data and load data are obtained by preprocessing the environmental time series data;

[0032] The temperature data, humidity data and load data are respectively clustered based on a preset dynamic clustering algorithm to obtain a temperature clustering center, a humidity clustering center and a load clustering center; wherein the temperature clustering center, the humidity clustering center and the load clustering center are respectively a temperature center, a humidity center and a load center;

[0033] The temperature clustering center, the humidity clustering center and the load clustering center are mapped to a two-dimensional plane based on a PCA algorithm to generate a temperature vertex, a humidity vertex and a load vertex;

[0034] A geometric triangular structure is constructed based on the temperature vertex, humidity vertex and load vertex as the vertices of the triangle.

[0035] The application maps the clustering centers of temperature, humidity and load to a two-dimensional plane based on the PCA algorithm to generate temperature vertices, humidity vertices and load vertices. Through PCA dimension reduction, not only the complexity of environmental data is simplified, but also the most representative features are retained, so that the model can more efficiently process these data. The introduction of the PCA algorithm makes the influence of environmental factors on the development of concrete cracks more intuitively presented, facilitating subsequent analysis and decision-making. In addition, by generating a geometric triangular structure based on the two-dimensional vertices obtained based on the PCA algorithm, and through the calculation of the influence area, the specific influence of temperature, humidity and load on the development of cracks is further quantified, improving the prediction accuracy.

[0036] Further, the preset dynamic clustering algorithm is used to cluster the temperature data, humidity data and load data respectively to obtain temperature clustering centers, humidity clustering centers and load clustering centers, including:

[0037] Corresponding temperature decay curves, humidity decay curves and load decay curves are generated based on the temperature data, humidity data and load data; wherein the temperature decay curves, humidity decay curves and load decay curves are constructed based on historical data and a preset exponential decay model;

[0038] Based on the preset dynamic clustering algorithm, the preset time window, the temperature decay curve, the humidity decay curve and the load decay curve, the temperature data, the humidity data and the load data are clustered respectively to obtain the temperature clustering centers, the humidity clustering centers and the load clustering centers.

[0039] By generating decay curves, the application can simulate the influence of temperature, humidity and load on concrete structures according to historical data and time lapse. The use of dynamic clustering algorithm and time window strategy makes the clustering process adapt to the changes of data in real time, improves the dynamicity and accuracy of the model, not only considers the influence of historical data, but also updates the clustering results according to real-time environmental changes, ensuring that the clustering centers always reflect the current environmental state.

[0040] Further, the PCA algorithm is used to map the temperature clustering centers, humidity clustering centers and load clustering centers to a two-dimensional plane to generate temperature vertices, humidity vertices and load vertices, including:

[0041] The temperature clustering centers, humidity clustering centers and load clustering centers are standardized based on a preset standardization algorithm to obtain standard temperature centers, standard humidity centers and standard load centers; the standard temperature centers, standard humidity centers and standard load centers are three-dimensional vectors;

[0042] construct a three-dimensional feature matrix based on the standard temperature center, the standard humidity center and the standard load center, and calculate a covariance matrix based on the three-dimensional feature matrix;

[0043] perform feature decomposition based on the covariance matrix, obtain three feature vectors, select two feature vectors as the base of a two-dimensional coordinate system, map the temperature clustering center, the humidity clustering center and the load clustering center to a two-dimensional plane, and generate a temperature vertex, a humidity vertex and a load vertex.

[0044] The present scheme of the application performs standardization processing on the clustering centers of temperature, humidity and load by the PCA algorithm, calculates a covariance matrix and performs feature decomposition, and finally maps to a two-dimensional plane, so that the two-dimensional representation of the clustering center can retain important information of the original data to the greatest extent, reduce redundancy, and ensure that the data after dimension reduction has higher interpretability and effectiveness. At the same time, through feature decomposition and covariance matrix calculation, it is ensured that the information of the clustering center after mapping to the two-dimensional plane is as complete as possible.

[0045] Further, the calculation of the temperature influence area, the humidity influence area and the load influence area based on the distance from the influence center to the edges of the geometric triangular structure comprises:

[0046] The distance from the influence center to the edges of the geometric triangular structure is calculated, and the temperature influence area, the humidity influence area and the load influence area are calculated based on the distance from the influence center to the edges of the geometric triangular structure and the length of the edges of the geometric triangular structure; wherein the area of the triangle formed by each vertex, the edge and the influence center is taken as the influence area of the vertex.

[0047] The application further quantifies the influence areas of temperature, humidity and load by calculating the distance from the influence center to the edges of the geometric triangular structure, and through the analysis of the geometric structure, the influence intensity of environmental factors on the concrete crack propagation can be more accurately evaluated, thereby providing a more reliable data basis for the prediction of the crack development trend, effectively converting the influence of environmental factors into quantifiable indicators, and improving the accuracy of crack prediction and decision basis.

[0048] Further, the splicing of the crack time-varying feature and the crack environmental feature to obtain a fusion feature, and the generation of a prediction result of the development trend of the concrete crack based on the fusion feature combined with a deep neural network, comprises:

[0049] The crack time-varying feature and the crack environmental feature are normalized to obtain standard time-varying features and standard environmental features, and the standard time-varying features and the standard environmental features are spliced to obtain a fusion feature;

[0050] The prediction result of the development trend of the concrete crack is generated based on the fusion feature combined with a deep neural network.

[0051] determine a risk level of the prediction result based on a preset trend grading table, and give a warning for the prediction result based on the risk level.

[0052] The application determines a risk level of the prediction result based on a preset trend grading table, and gives a warning for the prediction result based on the risk level.

[0053] In a second aspect, the application provides a concrete crack development trend prediction system based on time sequence images, comprising a data acquisition module, an expansion rate calculation module, a time-varying feature extraction module, a mapping module, a geometric construction module, an optimization module, an environmental feature extraction module and a prediction module.

[0054] The data acquisition module is configured to acquire high-definition images of the surface of the concrete structure, and construct a time sequence image dataset based on the high-definition images of the surface.

[0055] The expansion rate calculation module is configured to calculate the displacement vector of the crack edge pixel points of each image in the time sequence image dataset based on the optical flow method, and generate a crack expansion rate dataset.

[0056] The time-varying feature extraction module is configured to obtain crack time-varying features based on the crack expansion rate dataset and a preset crack trend prediction model.

[0057] The mapping module is configured to acquire environmental time sequence data of the concrete structure, determine temperature centers, humidity centers and load centers based on the environmental time sequence data, and map the temperature centers, humidity centers and load centers to a two-dimensional plane based on a PCA algorithm to generate temperature vertices, humidity vertices and load vertices.

[0058] The geometric construction module is configured to construct a geometric triangular structure based on the temperature vertices, humidity vertices and load vertices, take the barycenter of the geometric triangular structure as an influence center of the concrete structure, and calculate temperature influence areas, humidity influence areas and load influence areas based on the distances from the influence center to the edges of the geometric triangular structure.

[0059] The optimization module is configured to determine temperature weight, humidity weight and load weight based on the temperature influence area, humidity influence area and load influence area respectively, and to optimize a preset multi-modal fusion prediction network based on the temperature weight, humidity weight and load weight, to obtain a target multi-modal fusion prediction network.

[0060] The environment feature extraction module is configured to input the environment time series data into the target multi-modal fusion prediction network, to obtain crack environment features.

[0061] The prediction module is configured to splice the crack time-varying features and crack environment features to obtain fusion features, to generate a prediction result of a development trend of the concrete cracks based on the fusion features and a deep neural network, and to perform hierarchical early warning based on the prediction result, to complete the prediction of the development trend of the concrete cracks. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 FIG. 1 is a flowchart of a method for predicting a development trend of concrete cracks based on time series images according to an embodiment of the present application. DETAILED DESCRIPTION

[0063] The specific embodiments of the present application will be further described in detail below with reference to the accompanying drawings and embodiments. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0064] The terms "first" and "second" and the like in the specification and claims of the present application and the drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0065] In this document, the term "embodiment" means that the specific features, structures or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor is it independent or alternative to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0066] Embodiment 1

[0067] Reference is made to Figure 1 , Figure 1A flowchart of a concrete crack development trend prediction method based on time sequence images is provided for an embodiment of the present application. The embodiment of the present application provides a concrete crack development trend prediction method based on time sequence images, which comprises steps 101 to 107, and the details are as follows:

[0068] Step 101: Obtain a time sequence image dataset of a concrete structure, calculate displacement vectors of crack edge pixel points of each image in the time sequence image dataset based on an optical flow method, and generate a crack propagation rate dataset;

[0069] In the embodiment, the obtaining of the time sequence image dataset of the concrete structure, the calculation of the displacement vectors of the crack edge pixel points of each image in the time sequence image dataset based on the optical flow method, and the generation of the crack propagation rate dataset comprise:

[0070] Obtain a time sequence image dataset of a concrete structure, which is constructed from surface high-definition images of the concrete structure;

[0071] Based on a preset semantic segmentation algorithm and the time sequence image dataset, obtain a crack candidate region and a corresponding crack edge pixel set of each time sequence image;

[0072] Based on a preset point selection strategy, select feature points of each time sequence image in the crack edge pixel set;

[0073] For any adjacent time frames of time sequence images, calculate pixel-level displacement vectors of each feature point in the crack candidate region based on the optical flow method;

[0074] Based on the pixel-level displacement vectors, calculate physical displacements in adjacent time intervals, and generate a crack propagation rate dataset based on the physical displacements.

[0075] In the embodiment, surface high-definition images of the concrete structure are periodically collected by a high-definition camera device and high-frequency image collection, and the collection time stamps of the surface high-definition images are recorded. The surface high-definition images are preprocessed to eliminate blurred images, and a time sequence image dataset is constructed. The dataset contains series of information of concrete surface images at different times, and provides an image basis for subsequent crack monitoring and analysis.

[0076] In the embodiment, to ensure the accuracy of subsequent pixel-level displacement measurement, the surface of the target member is periodically photographed by a fixed or mobile high-definition camera, and the camera pose and environmental sensor time stamps are recorded at the same time.

[0077] In an optional embodiment, an industrial camera with a resolution not less than 12 MP or an equivalent high-definition camera (for example: 4000x3000 pixels) is used, and the frame rate is determined according to the crack development rate, and the commonly used values are 1 min, 10 min, 1 h or once a day; for high dynamic scenes, 1 fps can be taken. The camera installation must ensure that the field of view covers the crack area to be tested and is as parallel to the crack surface as possible. The installation method uses a tripod or a wall-mounted clamp, and at least three rigid reference marks are set at adjacent shooting points. All images are named by timestamp and stored in a unified metadata format, such as CSV / JSON format, which includes: file name, UTC timestamp, camera internal parameter reference ID, external parameter (pose) estimation or installation number, and environment sensor sampling time.

[0078] In this embodiment, the displacement vector of the crack edge pixel point in the time series image data is calculated by the optical flow method, and the crack expansion rate data set is further generated to accurately extract the crack candidate area and its edge pixels of each time series image, thereby providing high-quality input data for subsequent displacement vector calculation. By calculating the pixel-level displacement vector based on the optical flow method and converting it into physical displacement, this scheme can obtain the actual expansion of the crack in the adjacent time interval, avoiding the errors of manual intervention and subjective judgment. Moreover, by accurately calculating the pixel-level displacement, the expansion rate of the crack can be accurately quantified, providing key data for the prediction of the development trend of the crack, and realizing efficient and accurate monitoring of the expansion of the concrete crack, thereby providing a scientific basis for subsequent crack prediction and early warning.

[0079] In this embodiment, based on the preset semantic segmentation algorithm and the time series image data set, the crack candidate area and the corresponding crack edge pixel set of each time series image are obtained, including:

[0080] Detecting the time series image data set based on a preset target detection algorithm to obtain the crack candidate area of each time series image;

[0081] Extracting the crack binary mask of the crack candidate area of each time series image based on the preset semantic segmentation algorithm, and refining the crack binary mask to obtain the crack edge pixel set.

[0082] In this embodiment, after obtaining the time series image data set, a pre-trained semantic segmentation network is used to detect the crack area and extract the boundary of each frame of image.

[0083] In this embodiment, for each frame of high-definition image collected, a crack candidate region and a corresponding binary crack mask are obtained by a pre-trained semantic segmentation model. The preferred segmentation network is U-Net or DeepLabV3+, the input size is 1024x1024, and the training set is composed of manually annotated crack masks, and the annotation rules are specified in the specification.

[0084] In this embodiment, the model training parameter example: batch=8, lr=1e-4, optimizer Adam, training rounds 50-200, target validation set IoU ≥ 0.80.

[0085] In this embodiment, the crack binary mask output by the semantic segmentation is denoised and thinned by morphological opening and closing to obtain the crack centerline and edge pixel set; then sub-pixel edge coordinate set is obtained by sub-pixel boundary fitting (quadratic interpolation along gradient direction or sub-pixel interpolation based on Canny edge), and the corresponding confidence value is recorded. The crack candidate region and the crack edge pixel set are output together with the confidence, connected component ID and frame timestamp of each pixel, and the format example is a structured table or a JSON record.

[0086] In this embodiment, after obtaining the crack edge pixel set of each frame, a feature point set for cross-frame tracking is selected on the boundary according to a pre-set selection strategy. The selection strategy adopts a hybrid method: the first point set is obtained based on equidistant sampling of the crack skeleton; then the second point set is obtained by selecting curvature maximum points based on discrete curvature detection to capture corners and bifurcations; then the third point set is obtained by detecting strong corner points in the mask limited area based on corner detection to obtain texture stable points. Finally, the first point set, the second point set and the third point set are merged by de-duplication and priority to form the final feature point set; local descriptors are calculated for each feature point and local gradient, gray histogram and other information are recorded to improve the robustness of cross-frame matching.

[0087] In this embodiment, the upper limit of the total number of feature points is set to control the computational complexity, for example, no more than 200 points per square meter.

[0088] In this embodiment, for any pair of adjacent time frames, pixel-level tracking is performed on the feature point set within the crack candidate region to obtain displacement vectors. The preferred optical flow algorithm is Pyramidal LK or TV-L1 optical flow when the brightness / contrast changes greatly, and the algorithm parameters are: pyramid layer number 3, window 21x21, maximum iteration 30, convergence threshold 1e-2.

[0089] In this embodiment, to ensure tracking reliability, the forward displacement and the reverse displacement are calculated, and if + |>pre-set displacement threshold ε (ε is set to 1.0 pixel) to determine that the tracking fails and is rejected; at the same time, the descriptor matching score is combined as a secondary screening condition. In order to remove abnormal vectors and estimate local rigid deformation, an affine or rigid transformation model is fitted by RANSAC and vectors with residual exceeding the limit are removed.

[0090] In the embodiment, the crack candidate region in the time sequence image is extracted by the semantic segmentation algorithm and the target detection algorithm, and the crack edge pixel set is further obtained. Through the preset target detection algorithm, the crack region in the image can be automatically identified to ensure accurate positioning of the crack region, and then the binary mask of the crack region is extracted by the semantic segmentation algorithm, accurate crack edge pixel set is obtained through fine processing, the complexity and uncertainty in crack detection are solved, the error of manual labeling is avoided, and the automation and precision are improved.

[0091] In the embodiment, the physical displacement in the adjacent time interval is calculated based on the pixel-level displacement vector, and the crack propagation rate data set is generated based on the physical displacement, including:

[0092] The target displacement vector data is obtained by preprocessing the pixel-level displacement vector;

[0093] The target displacement vector data is converted into physical displacement by obtaining a preset camera calibration scale;

[0094] The local crack propagation rate is calculated based on the physical displacement in the adjacent time interval, and the time stamp corresponding to the local crack propagation rate is determined to generate the crack propagation rate data set.

[0095] In the embodiment, the pixel-level displacement is converted into physical displacement according to the camera calibration result or scene reference scale to generate the crack propagation rate data set. If the scene reference scale is used, the reference scale pixel length and the actual length are measured, and the physical displacement is obtained according to the reference scale pixel length and the actual length; if the depth information is available or stereo / structured light is used, the physical displacement is converted by the inverse projection formula of the camera intrinsic matrix K and the depth Z.

[0096] In the embodiment, the physical displacement is converted into crack propagation rate r = ΔX_mm / Δt (example output unit: mm / day, when the unit of Δt is second / hour, the corresponding conversion factor is used) according to the time interval Δt.

[0097] In this embodiment, in order to reduce noise, the physical displacement of the local feature points is filtered and interpolated by weighted average (distance weight or confidence weight) in space and exponential smoothing (smoothing coefficient a = 0.1-0.3) or Kalman filtering in time; the confidence interval and variance of each rate are recorded for subsequent risk assessment, and the final generated crack propagation rate dataset is stored in a structured table form. The crack propagation rate dataset serves as the input basis for the subsequent crack time-varying feature extraction and multi-modal fusion prediction network.

[0098] In this embodiment, by preprocessing the pixel-level displacement vector calculated by the optical flow method, obtaining the target displacement vector data, and converting it into physical displacement based on the camera calibration scale, the physical displacement in the adjacent time interval can be accurately calculated, and the crack propagation rate dataset can be further generated, which not only ensures the accuracy of the crack propagation rate, but also tracks the time evolution characteristics of the crack development through the record of the time stamp, further improving the accuracy of the prediction model.

[0099] Step 102: obtaining crack time-varying features based on the crack propagation rate dataset and a preset crack trend prediction model, wherein the crack trend prediction model is constructed based on an LSMT network;

[0100] In this embodiment, first, the crack propagation rate dataset is standardized and windowed according to the time sequence to form a number of sequence samples with a length of N; second, each sequence sample is input into the input layer of the LSMT model, and the model processes recursively according to the time step and outputs the hidden state at each time step; third, the time-varying feature vector corresponding to the sequence is obtained through the hidden state or through convergence (such as time-pooling, fully connected mapping, or attention weighting).

[0101] In this embodiment, the time-varying feature vector should at least include: the predicted rate sequence in the future period of time, the slope index representing the rate change trend, the fluctuation measure reflecting the short-term volatility, and the uncertainty measure for confidence evaluation; finally, the above time-varying features, corresponding time stamps, and original confidence information are stored in the time-varying feature set for subsequent splicing and fusion with environmental features.

[0102] In this embodiment, assuming that the sampling interval At = 1 hour and the sequence length N = 48 (i.e., inputting the data of the past 48 hours to predict the trend of the future 24 hours), the preprocessing includes: first, performing z-score standardization on each rate sequence to generate input samples.

[0103] In this embodiment, the model structure adopts a bidirectional LSTM (Bi-LSTM) stacked in two layers: the first layer of Bi-LSTM has 128 hidden units, and the second layer of Bi-LSTM has 64 hidden units; after the output of the second layer, a self-attention layer is connected to weight the time step hidden representation, and the attention output and the hidden state of the last time step are spliced to pass through two fully connected networks (FC1: 64→32 ReLU, FC2: 32→K) to obtain the prediction output, where K=24 represents the speed value of each hour in the next 24 hours.

[0104] Step 103: collecting environment time series data of the concrete structure, determining a temperature center, a humidity center and a load center based on the environment time series data, and constructing a geometric triangular structure based on the temperature center, the humidity center and the load center;

[0105] In this embodiment, the collecting environment time series data of the concrete structure, determining a temperature center, a humidity center and a load center based on the environment time series data, and constructing a geometric triangular structure based on the temperature center, the humidity center and the load center, comprises:

[0106] Collecting environment time series data of the concrete structure, pre-processing the environment time series data to obtain temperature data, humidity data and load data;

[0107] Based on a preset dynamic clustering algorithm, the temperature data, the humidity data and the load data are respectively clustered to obtain a temperature clustering center, a humidity clustering center and a load clustering center; wherein the temperature clustering center, the humidity clustering center and the load clustering center are respectively a temperature center, a humidity center and a load center;

[0108] Based on a PCA algorithm, the temperature clustering center, the humidity clustering center and the load clustering center are mapped to a two-dimensional plane to generate a temperature vertex, a humidity vertex and a load vertex;

[0109] Based on the temperature vertex, the humidity vertex and the load vertex as the vertices of a triangle, a geometric triangular structure is constructed.

[0110] In this embodiment, the collected environmental time series data is first aligned by time and pre-processed, including time synchronization, missing value interpolation, outlier detection and removal, and necessary filtering and denoising, to obtain temperature, humidity and load sequences for subsequent analysis. Then, the three types of sequences are clustered using the preset dynamic clustering algorithm, and the clustering results are represented by one or several representative cluster centers, which represent the typical state of the sequence within the set time window; each type of sequence only retains its main cluster center as the "cluster center" of that type (i.e. temperature cluster center, humidity cluster center, load cluster center). After obtaining the three cluster centers, they are mapped to the same feature space according to the unified standardization process (e.g. normalizing or z-score standardizing the original feature vectors of each cluster center), and two principal components are selected as the basis in the feature space based on the principal component analysis (PCA) method, and the three cluster centers are projected onto a two-dimensional plane to obtain three two-dimensional coordinate points (denoted as temperature vertex, humidity vertex, load vertex). Finally, the three two-dimensional vertices are used as the vertices of a triangle to construct a geometric triangular structure, and the centroid (as the impact center), the distance from the vertex to the edge, and the area of the sub-triangle formed by the centroid and the vertex are calculated on the triangular structure, so as to calculate the environmental impact area and derive the weight based on the area or distance in the subsequent analysis.

[0111] In this embodiment, the sampling frequency of environmental time series data is 1 hour, the analysis time window is the latest 30 days, the missing values are interpolated by linear interpolation (when the continuous missing is not more than 6 hours) or periodic interpolation based on the same time point of adjacent days (when the continuous missing is longer); the outliers are detected based on the median absolute deviation (MAD) method, the threshold is 3xMAD, the detected outliers are marked and processed by interpolation or manual review; and the original sequence is smoothed by exponential smoothing (example smoothing factor a=0.1) to reduce high-frequency noise, thereby generating temperature, humidity and load sequences.

[0112] In this embodiment, the cluster centers of temperature, humidity and load are mapped to a two-dimensional plane based on the PCA algorithm to generate temperature, humidity and load vertices. Through PCA dimension reduction, not only the complexity of environmental data is simplified, but also the most representative features are retained, making the model more efficient in processing these data. The introduction of PCA algorithm makes the influence of environmental factors on concrete crack development more intuitive, which is convenient for subsequent analysis and decision-making. In addition, the geometric triangular structure is generated based on the two-dimensional vertices obtained by the PCA algorithm, and the specific influence of temperature, humidity and load on crack development is further quantified through the calculation of impact area, which improves the prediction accuracy.

[0113] In this embodiment, the preset dynamic clustering algorithm is used to cluster the temperature data, humidity data and load data respectively, to obtain temperature clustering centers, humidity clustering centers and load clustering centers, including:

[0114] Based on the temperature data, humidity data and load data, corresponding temperature decay curves, humidity decay curves and load decay curves are generated; wherein the temperature decay curve, humidity decay curve and load decay curve are constructed based on historical data and a preset exponential decay model;

[0115] Based on the preset dynamic clustering algorithm, the preset time window, the temperature decay curve, the humidity decay curve and the load decay curve, the temperature data, the humidity data and the load data are clustered respectively to obtain the temperature clustering center, the humidity clustering center and the load clustering center.

[0116] In this embodiment, according to the collected environmental time series data, a corresponding decay curve is generated for each type of environmental quantity based on historical observation and a preset exponential decay model; the decay curve functions to assign different weights to samples at different time points in subsequent clustering, thereby reflecting the influence of the decay of historical contribution over time. Then, within a preset time window (window length can be configured), a feature vector (such as statistical quantities or combinations of frequency domain features such as mean, variance, recent peak, slope, etc.) is constructed for the decay-weighted samples, and the weighted features are used as input to call the preset dynamic clustering algorithm to cluster the temperature, humidity and load sequences respectively.

[0117] In this embodiment, environmental time series data of a concrete structure is collected, including temperature, humidity and load data. These data are collected by sensors at different time points to form a time series data set, reflecting the dynamic characteristics of environmental conditions changing over time. In order to better simulate the change process of environmental factors, before clustering, based on historical data and a preset exponential decay model, corresponding temperature decay curves, humidity decay curves and load decay curves are generated for temperature, humidity and load data respectively.

[0118] In this embodiment, an exponential decay model is used to describe the weakening process of the influence of each environmental factor on the structure. Specifically, the exponential decay model reflects the gradual weakening of the influence of environmental factors on the concrete structure over time through a decay factor λ. This decay factor can be fitted from historical data to ensure that the model is consistent with the actual situation. Through these decay curves, we can accurately simulate the trend of the influence of each environmental factor on crack propagation over time.

[0119] In this embodiment, the exponential decay model is:

[0120] (1)

[0121] where, is the impact intensity at time t (e.g., temperature, humidity, or load impact). is the impact intensity at initial time (i.e., t = 0. λ is the decay factor, controlling the rate of impact decay. A larger λ value means the impact decays faster. t is time, usually expressed in hours, days, or other suitable units.

[0122] In this embodiment, based on the preset dynamic clustering algorithm, combined with the time window and the decay curve, the temperature, humidity, and load data are clustered respectively. The dynamic clustering algorithm can dynamically adjust the position of the cluster center according to the change of environmental data, especially in time series data, as time goes on, the environmental conditions may change significantly, the clustering algorithm ensures that the cluster center can reflect the latest state of the environmental factor by introducing the time window. The time window strategy means that the clustering algorithm is not statically calculated once, but recalculates the cluster center in each time window according to the continuously updated data set, this way is more suitable for the dynamic changing data characteristics. Finally, the temperature clustering center, humidity clustering center, and load clustering center are obtained. These clustering centers represent the representative values of environmental factors affecting the concrete structure in different time periods. Each clustering center is the weighted mean of temperature, humidity, and load data in the corresponding time period, and through the weighting of the decay model, these clustering centers can more truly reflect the actual impact of environmental factors on the structure in different time periods.

[0123] In this embodiment, for each kind of environmental data (temperature, humidity, or load), first, based on the preset time window, it is divided into three time series. Each time series represents the change of environmental data in different time periods, this division method can capture the dynamic characteristics of environmental factors in different time periods. Specifically, the time window can be flexibly adjusted according to the actual monitoring needs, for example, selecting hourly, daily, or weekly time windows to reflect the changes of environmental factors in different time scales. Based on the preset clustering algorithm, the three time series are clustered. The selection of clustering algorithm can be based on the characteristics of data, common clustering algorithms such as K-means or hierarchical clustering, etc. Through the clustering algorithm, three typical values can be extracted from each time series, which represent the clustering center of environmental factors in this time period, that is, the most representative environmental factor value. In order to further improve the accuracy and reliability of the clustering result, combined with the decay curve, the data in each time series is weighted to ensure that the historical impact of environmental factors gradually weakens over time, and thus ensure that the clustering algorithm can reflect the actual impact of the current environmental conditions.

[0124] In this embodiment, after obtaining three typical values of each environmental data, the corresponding cluster centers are generated by combining the three typical values. The cluster center of each environmental data represents the weighted mean of the environmental factor within the time window, reflecting the typical change trend of the environmental factor in that period. For example, the cluster centers of temperature, humidity, and load respectively represent the typical values of each environmental factor in the three time periods, which can accurately describe the influence of environmental factors on concrete crack propagation.

[0125] In this embodiment, compared with traditional static clustering methods, it can effectively process time series data and consider the reversible change of environmental factors over time by introducing a decay model, with higher accuracy and adaptability. The application of this method in crack development trend prediction can make the model more accurately capture the change trend of crack propagation under different environmental conditions, providing more accurate basis for structural health monitoring and crack repair decision-making.

[0126] In this embodiment, by generating a decay curve, the influence of temperature, humidity, and load on concrete structures can be simulated according to historical data and time lapse. Using dynamic clustering algorithms and time window strategies, the clustering process can adapt to data changes in real time, improving the dynamicity and accuracy of the model. Not only does it consider the influence of historical data, but it also updates the clustering results according to real-time environmental changes, ensuring that the cluster centers always reflect the current environmental state.

[0127] In this embodiment, the PCA algorithm is used to map the temperature cluster center, humidity cluster center, and load cluster center into a two-dimensional plane to generate temperature vertices, humidity vertices, and load vertices, including:

[0128] The temperature cluster center, humidity cluster center, and load cluster center are standardized based on a preset standardization algorithm to obtain standard temperature centers, standard humidity centers, and standard load centers. The standard temperature centers, standard humidity centers, and standard load centers are three-dimensional vectors.

[0129] A three-dimensional feature matrix is constructed based on the standard temperature centers, standard humidity centers, and standard load centers, and a covariance matrix is calculated based on the three-dimensional feature matrix.

[0130] Based on the covariance matrix, feature decomposition is performed to obtain three feature vectors, and two feature vectors are selected as the basis of a two-dimensional coordinate system. The temperature cluster center, humidity cluster center, and load cluster center are mapped into a two-dimensional plane to generate temperature vertices, humidity vertices, and load vertices.

[0131] In this embodiment, the cluster centers of temperature, humidity, and load are input into the dimensionality reduction process as three-dimensional vectors. To ensure the comparability of the data and avoid computational bias due to different dimensions, the temperature cluster centers, humidity cluster centers, and load cluster centers are first standardized by a standardization algorithm. The purpose of the standardization process is to convert the data of each cluster center into a form with zero mean and unit variance, so that each environmental factor is compared at the same scale, ensuring that its contribution to subsequent analysis is equal.

[0132] In this embodiment, based on the standardized temperature center, humidity center, and load center, a three-dimensional feature matrix is constructed, where each row corresponds to a standardized cluster center vector of an environmental factor. This matrix is used to represent the relationship of environmental factors in three-dimensional space.

[0133] In this embodiment, the temperature, humidity, and load cluster center vectors are , and , and the three-dimensional feature matrix is specifically:

[0134] (2)

[0135] In this embodiment, based on this three-dimensional feature matrix, its covariance matrix is calculated, which reflects the mutual relationship between temperature, humidity, and load and their respective degrees of change. By calculating the covariance matrix, we can understand the variability of different environmental factors in the data and how they jointly change in space. After calculating the covariance matrix, eigen decomposition is performed to obtain three eigenvectors and corresponding eigenvalues. These eigenvectors represent the directions of maximum variance in the data, i.e., the main variation direction of the data.

[0136] In this embodiment, the first two eigenvectors are selected as the basis of the new two-dimensional coordinate system. These two eigenvectors can capture the most important change information in the data and map it to a two-dimensional space. By projecting the temperature cluster center, humidity cluster center, and load cluster center onto these two eigenvectors, we can obtain the two-dimensional coordinates of each environmental factor, thereby generating temperature vertices, humidity vertices, and load vertices. These vertices represent the positions of environmental factors in the two-dimensional plane, simplifying the representation of data and enabling a more intuitive reflection of the relative relationship between temperature, humidity, and load.

[0137] In this embodiment, the process effectively maps three-dimensional data into two-dimensional space while preserving the main information of the data, thereby providing simplified and effective input features for subsequent concrete crack development trend prediction. Through the PCA algorithm, the clustering centers of temperature, humidity, and load are not only more computationally efficient, but also make the model's performance in environmental factor analysis more intuitive and easier to understand.

[0138] In this embodiment, the scheme standardizes the clustering centers of temperature, humidity, and load through the PCA algorithm, calculates the covariance matrix, and performs feature decomposition, finally mapping to a two-dimensional plane. This allows the two-dimensional representation of the clustering centers to retain the important information of the original data to the greatest extent, reduce redundancy, and ensure that the reduced data has higher interpretability and effectiveness. At the same time, through feature decomposition and covariance matrix calculation, it ensures that the information of the clustering centers after mapping to the two-dimensional plane is as complete as possible.

[0139] Step 104: Taking the center of gravity of the geometric triangular structure as the impact center of the concrete structure, and calculating the temperature impact area, humidity impact area, and load impact area based on the distance from the impact center to each side of the geometric triangular structure;

[0140] In this embodiment, the calculation of the temperature impact area, humidity impact area, and load impact area based on the distance from the impact center to each side of the geometric triangular structure includes:

[0141] The distance from the impact center to each side of the geometric triangular structure is calculated, and the temperature impact area, humidity impact area, and load impact area are calculated based on the distance from the impact center to each side of the geometric triangular structure and the length of each side of the geometric triangular structure. The area of the triangle formed by each vertex, the side, and the impact center is taken as the impact area of that vertex.

[0142] In this embodiment, the clustering centers of temperature, humidity, and load are mapped to a two-dimensional plane through the PCA algorithm, and a geometric triangular structure is used to represent the relative relationship between temperature, humidity, and load. The geometric triangular structure is composed of three vertices corresponding to the clustering centers of temperature, humidity, and load.

[0143] In this embodiment, a triangular structure composed of the temperature cluster center, the humidity cluster center and the load cluster center on a two-dimensional plane is determined, and the lengths of the three sides of the triangle and the coordinates of the three vertices of the triangle are calculated; secondly, according to the determined influence center position, the perpendicular distances of the influence center to the three sides of the triangular structure are calculated respectively (that is, the shortest distance of the influence center to each side is calculated); then, the area of the small triangle composed of each side and the influence center is calculated as the influence area of the vertex opposite to the side according to the conventional geometric method, taking each side as the base and the corresponding perpendicular distance as the height (for example, the influence area corresponding to the temperature vertex is determined by the perpendicular distance of the influence center to the humidity side and the length of the side).

[0144] In this embodiment, the influence center of the triangular structure ABC composed of the temperature cluster center A, the humidity cluster center B and the load cluster center C on a two-dimensional plane is P, wherein the influence area of the temperature cluster center A is , the influence area of the humidity cluster center B is , and the influence area of the load cluster center C is .

[0145] In this embodiment, geometrically, each influence area is equal to half of the product of the length of the corresponding base and the perpendicular distance of the influence center to the base, thus reflecting both the scale of the mode represented by the base (the length of the base) in the overall configuration and the geometric distance of the influence center (the perpendicular distance). If the influence center is close to a certain base (i.e. far from the opposite vertex), the corresponding influence area is smaller, and thus the opposite vertex (representing the mode) has a greater contribution to the current influence point; vice versa.

[0146] In this embodiment, by calculating the distances of the influence center to the sides of the geometric triangular structure, the influence areas of temperature, humidity and load are further quantified. Through the analysis of the geometric structure, the influence intensity of environmental factors on concrete crack propagation can be more accurately evaluated, thus providing a more reliable data basis for crack development trend prediction, effectively converting the influence of environmental factors into quantifiable indicators, and improving the accuracy of crack prediction and decision basis.

[0147] Step 105: determining a temperature weight, a humidity weight and a load weight based on the temperature influence area, the humidity influence area and the load influence area respectively, and optimizing a preset multi-modal fusion prediction network based on the temperature weight, the humidity weight and the load weight to obtain a target multi-modal fusion prediction network;

[0148] In the embodiment, the three influence areas are normalized as weights, equivalent to using the barycentric representation of a triangle, the position of the influence point P in the triangle ABC is represented by a set of non-negative and sum-to-one coefficients, which ensures that the resulting weights are convex combinations, numerically stable and directly applicable as weighted coefficients in multi-modal fusion, so that the relative importance of each environmental factor under different working conditions can be intuitively understood through geometric figures; and with the slight movement of P, the weights change continuously, avoiding the discontinuity caused by discrete mapping, thereby facilitating the stability of model training and the robustness of reasoning; at the same time, both the modal "intensity" and "spatial correlation" information are taken into account, so that the weights depend not only on the absolute value of a single variable, but also reflect the coupling relationship of multiple factors in the spatial / working condition configuration.

[0149] In the embodiment, the three influence areas are normalized to obtain temperature weight , humidity weight and load weight , i.e. each influence area is divided by the sum of the three to ensure that the weight is non-negative and sum-to-one, and a small positive regularization term is added in the denominator if necessary to prevent numerical instability. Then, the obtained weights are used as prior information of environmental modal in the multi-modal fusion prediction network for network optimization.

[0150] In the embodiment, based on the weights, the multi-modal fusion prediction network includes: multiplying the temperature, humidity and load channels at the modal input layer of the network by the corresponding weights , , respectively to achieve preliminary amplification or suppression of the importance of different environmental channels; further, introducing , , in the form of logarithmic bias or gating coefficient on the corresponding logits of the cross-modal attention or gating unit, so that the data-driven attention allocation starts with geometric prior and is dynamically adjusted in training; finally, in the training target, the sub-loss term related to environmental consistency or environment is imposed , , To weight the coefficients, the training process pays more attention to the environmental modalities that are important to the geometric decision while optimizing the overall prediction performance. In implementation, offline pre-training is preferred: preliminary learning is completed on a baseline model without using weights or using equal weights, and then the network is fine-tuned with the obtained geometric weights, the effectiveness of the weights is evaluated by cross-validation, and overfitting is prevented by early stopping, learning rate scheduling, and regularization; in the online stage, the cluster center and the influence center are recalculated through a sliding time window or event triggering, and the weights are updated, so that the time-varying adaptive weights are realized, and the confidence threshold is used to control whether to perform model fine-tuning to ensure system stability. To ensure engineering robustness, in the case of triangular degeneration or extremely small area, the system should switch to a backup strategy (such as inverse weight based on the distance between the centroid and the vertex or pre-defined empirical weight), and uniform coordinate scale and unit calibration should be performed before mapping and weight calculation, and sensor confidence should be fused and weighted to reduce the influence of observation error on weight and network performance. By using the above optimization method based on area normalization weight and synchronously applied in input, attention, and loss, etc., the sensitivity of the multi-modal fusion network to key environmental factors under different working conditions can be improved, the convergence speed can be accelerated, and the physical interpretability and engineering applicability of the prediction output can be enhanced.

[0151] Step 106: inputting the environmental time series data into the target multi-modal fusion prediction network to obtain a crack environmental feature;

[0152] In this embodiment, the multi-modal fusion network includes an environmental channel, which is responsible for the environmental encoder: the normalized and time-aligned temperature, humidity, and load sequences are first extracted for local time series representation through several one-dimensional convolutions or small LSTMs, and then low-dimensional environmental embeddings are obtained through fully connected layers; the normalized weight vector calculated based on the geometric centroid of the triangle is introduced at the environmental encoder , , As an input gate, the embeddings of the temperature, humidity, and load channels are scaled or used as gating coefficients to realize prior adjustment of the importance of the environmental channel. The fusion module uses a cross-modal multi-head attention mechanism, taking the crack time-varying feature as the query (Q), the weighted environmental embedding feature as the key / value (K / V), and injecting the weight prior (for example, in the form of a logarithmic bias or a learnable scaling factor) into the attention logits to guide attention allocation; the attention output is stabilized after residual connection and layer normalization and sent to a number of layer fusion fully connected networks (MLP, including BatchNorm and dropout) to obtain a unified fusion representation, i.e., the crack environmental feature.

[0153] Step 107: splicing the crack time-varying features and crack environment features to obtain fused features, and generating a prediction result of the development trend of the concrete cracks based on the fused features combined with a deep neural network;

[0154] In this embodiment, the splicing of the crack time-varying features and crack environment features to obtain fused features, and the generation of a prediction result of the development trend of the concrete cracks based on the fused features combined with a deep neural network, comprises:

[0155] The crack time-varying features and crack environment features are normalized to obtain standard time-varying features and standard environment features, and the standard time-varying features and standard environment features are spliced to obtain fused features;

[0156] The prediction result of the development trend of the concrete cracks is generated based on the fused features combined with a deep neural network;

[0157] The risk level of the prediction result is determined based on a preset trend grading table, and the prediction result is warned based on the risk level.

[0158] In this embodiment, the crack time-varying features (such as local displacement, length, width and their rate sequences obtained by optical flow and LSTM) and environment features (temperature, humidity and load sequences adjusted by geometric triangle normalization weight) from the previous step are strictly aligned in time and space, and the processing includes timestamp synchronization, missing value interpolation and outlier removal; then the standardized processing is performed on each channel feature after alignment (Z-score standardization or Min-Max normalization can be used, and different normalization schemes can be used for different channels to retain the physical meaning if necessary), to obtain standard time-varying features and standard environment features.

[0159] In this embodiment, the standardized time-varying features and environment features are spliced into a fusion feature vector (a fusion feature matrix of time series can be constructed for sequence tasks) in a predetermined order at each time or in each time window, and a small fully connected layer (projection layer) can be optionally applied after splicing for dimension matching and preliminary nonlinear transformation. Based on the fused features, a deep neural network is used to generate a prediction result of the development trend of the concrete cracks: the network structure is preferably a combination of "fusion encoder and time series decoder", the encoder is composed of several layers of fully connected (including BatchNorm, activation and Dropout) or Transformer encoding layers to extract cross-modal correlation, and the decoder uses LSTM / Transformer decoder or multi-step regression head to output the crack length, width and expansion rate sequence at future time points; at the same time, a classification branch is set in parallel to output the warning level distribution and estimate the uncertainty (such as obtaining the confidence interval by MC-dropout or ensemble method).

[0160] In this embodiment, after obtaining the prediction result, the continuous prediction value is mapped to a risk level (such as normal, warning, danger, or more detailed levels) according to a preset trend grading table (which can be a threshold table set based on engineering experience or a risk grading model adapted and fitted from historical data), and a corresponding early warning action (including but not limited to local / cloud logging, sending SMS / email / APP push, generating a visual report and recommended treatment measures) is triggered according to the mapped risk level and the prediction confidence.

[0161] To ensure the stability and engineering applicability of the model, staged training (pre-training offline and then fine-tuning with geometric weights), cross-validation, and early stopping strategies are used during training, and in online deployment, sliding window recalculation of clustering and influence center, and confidence threshold determination of whether to perform model fine-tuning or manual review are used; when the model uncertainty is high or the triangular weight is abnormal, the system should reduce the priority of automatic treatment and issue a manual inspection notification. The above implementation methods take into account numerical stability, physical interpretability, and real-time engineering application, and can provide reliable crack development prediction and graded early warning support for structural health monitoring under different working conditions.

[0162] In this embodiment, the risk level of the prediction result is determined based on a preset trend grading table, and the prediction result is warned based on the risk level. In this embodiment, by splicing the time-varying characteristics of the crack and the environmental characteristics, the prediction result of the development trend of the concrete crack is generated by combining the deep neural network. Through the learning ability of the deep neural network, the model can automatically extract complex nonlinear relationships from the fused features, thereby improving the accuracy of crack prediction. Based on this prediction result, graded early warning can timely identify the development trend of cracks at different risk levels, ensure that engineering personnel can quickly respond and take appropriate repair measures, reduce safety risks, and prolong the service life of the concrete structure.

[0163] In this embodiment, by collecting high-definition images of the surface of the concrete structure and constructing a time-series dataset based on the time-series images, the evolution process of the cracks can be comprehensively and dynamically tracked. By calculating the displacement vector of the crack edge pixel points in the time-series image data through the optical flow method, a crack propagation rate dataset is generated. This process can accurately quantify the propagation of the cracks, providing basic data for further prediction of the development trend of the cracks. It can automatically and continuously track the development of the cracks, avoiding the errors and omissions of traditional manual detection, and improving the monitoring accuracy and efficiency. Moreover, by introducing a crack trend prediction model, the time-varying characteristics can be deeply mined, and the dynamic rules in the evolution process of the cracks can be captured, providing a prediction basis for the future trend of the cracks. The time-varying characteristics of the cracks not only reflect the current state of the cracks, but also can predict the possible future trend of the cracks through historical data analysis, providing decision support for long-term structure maintenance and reinforcement. Furthermore, by collecting and processing the environmental time-series data of the concrete structure, a geometric triangle structure based on the temperature vertex, humidity vertex and load vertex is constructed, and the center of gravity of the geometric triangle structure is taken as the influence center of the concrete structure, effectively quantifying the comprehensive influence of different environmental factors on the development of the cracks. By calculating the influence area of temperature, humidity and load based on the distance from the influence center to each side of the geometric triangle, the influence intensity of each environmental factor on the development of the cracks in the concrete structure is reflected. Specifically, the temperature influence area, humidity influence area and load influence area can assign a weight to each environmental factor, and these weights reflect the importance of different environmental factors in the development process of the cracks in the concrete structure. Based on the three weights as the optimization target input into the multi-modal fusion prediction network, the prediction accuracy of the network is further improved. The introduction of the weights not only considers the independent influence of each environmental factor, but also comprehensively considers the mutual relationship between them, so that the prediction network can more accurately capture the dynamic changes of different environmental factors on the development trend of the cracks in the concrete structure. The model not only improves the prediction ability of the development trend of the cracks in the concrete structure, but also enhances the adaptability and accuracy of the model to complex environmental conditions. On this basis, the crack time-varying characteristics and the crack environmental characteristics are spliced to form a fusion feature based on the deep neural network, and the prediction result of the development trend of the cracks is generated based on the fusion feature. It can learn and extract complex data relationships, and improve the prediction accuracy.

[0164] The embodiment of the present application also provides a concrete crack development trend prediction system based on time-series images, which comprises a data acquisition module, an expansion rate calculation module, a time-varying characteristic extraction module, a mapping module, a geometric construction module, an optimization module, an environmental characteristic extraction module and a prediction module.

[0165] The data acquisition module is used for collecting high-definition images of the surface of the concrete structure, and constructing a time-series image dataset based on the high-definition images of the surface.

[0166] The expansion rate calculation module is configured to calculate displacement vectors of crack edge pixel points of each image in the time-series image dataset based on an optical flow method, and generate a crack expansion rate dataset;

[0167] The time-varying feature extraction module is configured to obtain crack time-varying features based on the crack expansion rate dataset and a preset crack trend prediction model, wherein the crack trend prediction model is constructed based on an LSMT network.

[0168] The mapping module is configured to collect environment time-series data of the concrete structure, determine a temperature center, a humidity center and a load center based on the environment time-series data, and map the temperature center, the humidity center and the load center to a two-dimensional plane based on a PCA algorithm to generate a temperature vertex, a humidity vertex and a load vertex.

[0169] The geometric construction module is configured to construct a geometric triangular structure based on the temperature vertex, the humidity vertex and the load vertex, take a barycenter of the geometric triangular structure as an influence center of the concrete structure, and calculate a temperature influence area, a humidity influence area and a load influence area based on distances from the influence center to edges of the geometric triangular structure.

[0170] The optimization module is configured to determine a temperature weight, a humidity weight and a load weight based on the temperature influence area, the humidity influence area and the load influence area respectively, and optimize a preset multi-modal fusion prediction network based on the temperature weight, the humidity weight and the load weight to obtain a target multi-modal fusion prediction network.

[0171] The environment feature extraction module is configured to input the environment time-series data into the target multi-modal fusion prediction network to obtain crack environment features.

[0172] The prediction module is configured to splice the crack time-varying features and the crack environment features to obtain fusion features, generate a prediction result of a concrete crack development trend based on the fusion features in combination with a deep neural network, and perform hierarchical early warning based on the prediction result to complete the prediction of the concrete crack development trend.

[0173] In the embodiment of the present application, a terminal is also provided, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the above-mentioned concrete crack development trend prediction method based on time-series images when executing the computer program.

[0174] In the embodiment of the present application, a computer readable storage medium is also provided, which includes a stored computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to execute the above-mentioned concrete crack development trend prediction method based on time-series images when the computer program runs.

[0175] For example, the computer program can be divided into one or more modules, one or more modules are stored in the memory and executed by the processor to complete the present application. One or more modules can be a series of computer program instructions capable of completing a specific function, which is used to describe the execution process of the computer program in the terminal.

[0176] The terminal can be a desktop computer, notebook, palm computer and cloud server and other computing devices. The terminal can include, but not limited to, processor, memory, display. Those skilled in the art can understand that the above components are only examples of the terminal and do not constitute a limitation on the terminal, which can include more or less components, or combine certain components, or different components, for example, the terminal can also include input / output devices, network access devices, bus, etc.

[0177] The processor can be a central processing unit (CPU), but also other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can be any conventional processor, etc. The processor is the control center of the terminal, which connects all parts of the terminal through various interfaces and lines.

[0178] The memory can be used to store computer programs and / or modules, and the processor can realize various functions of the terminal by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function (such as a sound playing function, a character conversion function, etc.), etc.; the data storage area can store data created according to the use of the terminal (such as audio data, text message data, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0179] The above-described specific embodiments further illustrate the objects, technical solutions, and advantages of the present application. It should be understood that the above-described specific embodiments are merely for the purpose of illustrating the present application, and are not intended to limit the protection scope of the present application. It is particularly pointed out that any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for predicting the development of concrete cracks based on time-series images, characterized by, The method comprises the following steps: acquiring a time-series image data set of a concrete structure, calculating displacement vectors of crack edge pixel points of each image in the time-series image data set based on an optical flow method, and generating a crack propagation rate data set; based on the crack propagation rate data set and a preset crack trend prediction model, obtaining a crack time-varying feature; the crack trend prediction model is constructed based on an LSMT network; collecting environmental time-series data of the concrete structure, determining a temperature center, a humidity center and a load center based on the environmental time-series data, and constructing a geometric triangular structure based on the temperature center, the humidity center and the load center; taking the barycenter of the geometric triangular structure as an influence center of the concrete structure, and calculating a temperature influence area, a humidity influence area and a load influence area based on the distances from the influence center to the sides of the geometric triangular structure; determining a temperature weight, a humidity weight and a load weight based on the temperature influence area, the humidity influence area and the load influence area respectively, and optimizing a preset multi-modal fusion prediction network based on the temperature weight, the humidity weight and the load weight, to obtain a target multi-modal fusion prediction network; inputting the environmental time-series data into the target multi-modal fusion prediction network to obtain a crack environmental feature; splicing the crack time-varying feature and the crack environmental feature to obtain a fusion feature, and generating a prediction result of a development trend of a concrete crack based on the fusion feature combined with a deep neural network.

2. The method of claim 1, wherein the method comprises: The method comprises the following steps: acquiring a time-series image data set of a concrete structure, calculating displacement vectors of crack edge pixel points of each image in the time-series image data set based on an optical flow method, and generating a crack propagation rate data set; acquiring a time-series image data set of a concrete structure, the time-series image data set being constructed based on a high-definition image of a surface of the concrete structure; obtaining a crack candidate region and a corresponding crack edge pixel set of each time-series image based on a preset semantic segmentation algorithm and the time-series image data set; selecting feature points of each time-series image in the crack edge pixel set based on a preset selection strategy; for any adjacent time frames of time-series images, calculating pixel-level displacement vectors of each feature point in the crack candidate region based on the optical flow method; 3. The method of claim 2, wherein the method further comprises: calculating physical displacement in an adjacent time interval based on the pixel-level displacement vectors, and generating a crack propagation rate data set based on the physical displacement. The method comprises the following steps: detecting the time-series image data set based on a preset target detection algorithm to obtain a crack candidate region of each time-series image; 4. The method of claim 3, wherein the method further comprises: extracting a crack binary mask of the crack candidate region of each time-series image based on a preset semantic segmentation algorithm, and performing thinning processing on the crack binary mask to obtain a crack edge pixel set. The method comprises the following steps: preprocessing the pixel-level displacement vectors to obtain target displacement vector data; obtaining a preset camera calibration scale to convert the target displacement vector data into physical displacement; calculating a local crack propagation rate based on the physical displacement in adjacent time intervals, and determining a timestamp corresponding to the local crack propagation rate to generate a crack propagation rate dataset.

5. The method of claim 4, wherein the method further comprises: The environment time series data of the concrete structure is collected, and the temperature center, humidity center and load center are determined based on the environment time series data, and a geometric triangular structure is constructed based on the temperature center, humidity center and load center, including: Collecting the environment time series data of the concrete structure, preprocessing the environment time series data to obtain temperature data, humidity data and load data; Based on the preset dynamic clustering algorithm, the temperature data, humidity data and load data are clustered respectively to obtain temperature clustering center, humidity clustering center and load clustering center; wherein the temperature clustering center, humidity clustering center and load clustering center are temperature center, humidity center and load center respectively; Based on the PCA algorithm, the temperature clustering center, humidity clustering center and load clustering center are mapped into a two-dimensional plane to generate temperature vertex, humidity vertex and load vertex; Based on the temperature vertex, humidity vertex and load vertex as the vertices of the triangle, a geometric triangular structure is constructed.

6. The method of claim 5, wherein the method further comprises: The temperature data, humidity data and load data are clustered based on the preset dynamic clustering algorithm to obtain temperature clustering center, humidity clustering center and load clustering center, including: Based on the temperature data, humidity data and load data, corresponding temperature decay curve, humidity decay curve and load decay curve are generated; wherein the temperature decay curve, humidity decay curve and load decay curve are constructed based on historical data and a preset exponential decay model; Based on the preset dynamic clustering algorithm, the temperature data, humidity data and load data are clustered based on the preset time window combined with the temperature decay curve, humidity decay curve and load decay curve to obtain the temperature clustering center, humidity clustering center and load clustering center.

7. The method of claim 6, wherein the method further comprises: The temperature clustering center, humidity clustering center and load clustering center are mapped into a two-dimensional plane based on the PCA algorithm to generate temperature vertex, humidity vertex and load vertex, including: Based on the preset standardization algorithm, the temperature clustering center, humidity clustering center and load clustering center are standardized to obtain standard temperature center, standard humidity center and standard load center; the standard temperature center, standard humidity center and standard load center are three-dimensional vectors; Based on the standard temperature center, standard humidity center and standard load center, a three-dimensional feature matrix is constructed, and a covariance matrix is calculated based on the three-dimensional feature matrix; Based on the covariance matrix, feature decomposition is performed to obtain three feature vectors, and two feature vectors are selected as the basis of a two-dimensional coordinate system to map the temperature clustering center, humidity clustering center and load clustering center into a two-dimensional plane to generate temperature vertex, humidity vertex and load vertex.

8. The time-lapse image-based concrete crack development trend prediction method of claim 1, wherein, The temperature influence area, humidity influence area and load influence area are calculated based on the distance from the influence center to each side of the geometric triangular structure, including: The distances from the influence center to the edges of the geometric triangular structure are calculated, and the temperature influence area, the humidity influence area and the load influence area are calculated based on the distances and the lengths of the edges of the geometric triangular structure; wherein the area of the triangle formed by each vertex pair edge and the influence center is taken as the influence area of the vertex.

9. A time-lapse image-based concrete crack development trend prediction method according to any one of claims 1 to 8, characterized in that, The time-varying feature and the crack environment feature are spliced to obtain a fusion feature, and a prediction result of a development trend of the concrete crack is generated based on the fusion feature and a deep neural network. The time-varying feature and the crack environment feature are normalized to obtain standard time-varying features and standard environment features, and the standard time-varying features and the standard environment features are spliced to obtain a fusion feature. A prediction result of a development trend of the concrete crack is generated based on the fusion feature and a deep neural network. The risk level of the prediction result is determined based on a preset trend grading table, and the prediction result is warned based on the risk level.

10. A time-lapse image-based concrete crack development trend prediction system, comprising: It comprises: an expansion rate calculation module, a time-varying feature extraction module, a geometric construction module, an area calculation module, an optimization module, an environment feature extraction module and a prediction module; The expansion rate calculation module is configured to obtain a time sequence image data set of a concrete structure, calculate displacement vectors of crack edge pixel points of each image in the time sequence image data set based on an optical flow method, and generate a crack expansion rate data set. The time-varying feature extraction module is configured to obtain a crack time-varying feature based on the crack expansion rate data set and a preset crack trend prediction model, wherein the crack trend prediction model is constructed based on an LSMT network. The geometric construction module is configured to collect environment time sequence data of the concrete structure, determine a temperature center, a humidity center and a load center based on the environment time sequence data, and construct a geometric triangular structure based on the temperature center, the humidity center and the load center. The area calculation module is configured to take the barycenter of the geometric triangular structure as an influence center of the concrete structure, and calculate a temperature influence area, a humidity influence area and a load influence area based on distances from the influence center to edges of the geometric triangular structure. The optimization module is configured to determine a temperature weight, a humidity weight and a load weight based on the temperature influence area, the humidity influence area and the load influence area respectively, and optimize a preset multi-modal fusion prediction network based on the temperature weight, the humidity weight and the load weight to obtain a target multi-modal fusion prediction network. The environment feature extraction module is configured to input the environment time sequence data into the target multi-modal fusion prediction network to obtain a crack environment feature. The prediction module is configured to splice the crack time-varying feature and the crack environment feature to obtain a fusion feature, and generate a prediction result of a development trend of the concrete crack based on the fusion feature and a deep neural network.

Images