Early warning method and device for concrete structure crack

Abstract

The invention provides an early warning method and device for concrete structure cracks, and relates to the technical field of civil engineering structure detection. The method comprises the steps of obtaining a multi-view two-dimensional image, a three-dimensional laser point cloud and a sensor pose of a concrete structure at different time points, performing improved box counting fractal dimension analysis on each frame, generating a pixel-level fractal matrix, embedding the pixel-level fractal matrix into a crack segmentation model, fusing the matrix and the image to complete crack mask output, and obtaining a crack mask. And carrying out multi-source fusion on the mask and the point cloud, and carrying out spatial registration to obtain a three-dimensional point cloud model of the marked crack. Clustering and main axis fitting are carried out on crack points, geometric parameters such as a space trajectory, width and depth are calculated, and multi-temporal evolution analysis is carried out based on a three-dimensional model to construct an evolution trajectory model and time sequence characteristics. And finally, structure safety early warning information is generated through an evaluation module. According to the method, accurate crack detection under the zero sample condition is realized, and the future development trend is predicted by using the time sequence model.

Description

Technical Field

[0001] This invention relates to the field of civil engineering structural testing technology, and in particular to a method and device for early warning of cracks in concrete structures. Background Technology

[0002] Currently, most concrete crack detection methods rely on two-dimensional images, which leads to problems such as insufficient spatial positioning accuracy of cracks, large errors in width estimation, difficulty in adapting to curved structures, and lack of continuous tracking over time. Although some studies have attempted to map image information onto three-dimensional models, these methods usually require targets, have low mapping accuracy, and are not suitable for multi-temporal crack evolution analysis. The detection and assessment of cracks in concrete structures is a crucial link in structural health monitoring and safety early warning. However, traditional crack detection mainly relies on manual inspection or contact instrument measurement, which is not only labor-intensive and inefficient but also significantly affected by human factors, making it difficult to meet the needs of modern large-scale, high-frequency, non-contact detection.

[0003] With the development of computer vision and 3D reconstruction technologies, researchers have begun to utilize image segmentation algorithms to identify crack morphology, and then combine this with LiDAR scanning to construct 3D models of the cracks, enabling spatial localization and size quantification. However, existing methods still have many shortcomings. First, the fusion of 2D images and 3D information is insufficient: many algorithms simply map the crack outline in the image onto the point cloud surface, lacking necessary pose constraints and semantic consistency, resulting in low spatial localization accuracy. Second, these methods are poorly adaptable to complex curved structures: on surfaces with large curvature, such as bridge piers and tunnel linings, traditional projection algorithms often produce severe deformation or missed detections. Furthermore, most current research lacks the ability to model the temporal dimension, usually focusing only on single detection results, making it difficult to capture the expansion and evolution characteristics of cracks during the structural service life, thus failing to provide a basis for long-term health management. Finally, crack quantification indicators are relatively singular, often focusing only on maximum width or apparent length, neglecting changes in the internal morphology of the crack and the comprehensive analysis of multi-scale profile information. Therefore, there is an urgent need for a comprehensive crack detection method that integrates semantic recognition, 3D localization, size quantification, and temporal tracking.

[0004] This method should be able to robustly identify cracks of arbitrary shapes even in the absence of crack-specific training data, thereby improving detection accuracy; it should also fuse images with multi-source information such as point clouds to achieve accurate three-dimensional reconstruction of cracks; and it should have the ability to conduct long-term, multiple observations and comparisons of the same crack, extract evolution patterns, and predict development trends, thus providing a scientific basis for long-term infrastructure maintenance decisions. Summary of the Invention

[0005] To address the technical problem of existing technologies being unable to accurately detect and predict cracks under zero-sample conditions, this invention provides a method and apparatus for early warning of cracks in concrete structures. The technical solution is as follows:

[0006] On the one hand, a method for early warning of cracks in concrete structures is provided, which is implemented by a crack early warning device for concrete structures, and the method includes: S1: Acquire multi-view two-dimensional digital image sequences of the concrete structure to be inspected at different time points, the corresponding three-dimensional laser point cloud, and the sensor spatial pose information recorded at each time point simultaneously to obtain the original data set of the concrete structure to be inspected. S2: Perform fractal dimension analysis on the data of the original dataset of the concrete structure to be detected at each time step to obtain the pixel-level fractal dimension matrix at each time step. The fractal dimension analysis includes: based on the improved box counting algorithm, introducing orientation sensitivity and local adaptive window mechanism, calculating the texture complexity of the neighborhood of each pixel in the image, and simultaneously counting the minimum number of boxes containing crack pixels under different scales and orientations of grid coverage. The pixel-level fractal dimension matrix is ​​used as a prompt information to be embedded in the feature layer of the image segmentation model. S3: The pixel-level fractal dimension matrix at each time step and the corresponding image in the multi-view two-dimensional digital image sequence are fused and input into the pre-trained crack segmentation model to perform crack segmentation, and the crack mask image at each time step is output. S4: Perform multi-source data fusion and 3D reconstruction on the crack mask image and the corresponding 3D laser point cloud at each time moment to obtain a set of crack points that are projected, mapped and marked in the 3D laser point cloud. The multi-source data fusion and 3D reconstruction includes spatial registration of the processing results of all image frames to form a 3D point cloud model containing complete crack information. S5: Based on the three-dimensional point cloud model, clustering and main axis fitting calculations are performed on the set of crack points to obtain the crack spatial trajectory and crack width. The clustering and main axis fitting calculations include estimating geometric parameters such as crack depth to form a three-dimensional crack model at each moment. The crack width is obtained by calculating the maximum distance between crack points in the vertical direction of the crack spatial trajectory. S6: Perform multi-temporal crack evolution analysis on the three-dimensional crack model at each time point to quantify temporal characteristics, and finally construct a crack evolution trajectory model. The temporal characteristics include crack length growth, crack width variation, and crack fractal dimension variation. S7: Input the time series features into the pre-trained time series model to obtain the predicted values ​​of crack length and crack width. Input the predicted values ​​of crack length and crack width into the prediction result evaluation module to obtain early warning information. The prediction result evaluation module includes evaluating the structural safety status based on the prediction results and providing data support for subsequent maintenance decisions. The pre-trained time series model is constructed based on a long short-term memory neural network.

[0007] Preferably, step S1 involves acquiring a multi-view two-dimensional digital image sequence of the concrete structure to be inspected at different time points, the corresponding three-dimensional laser point cloud, and the synchronously recorded sensor spatial pose information at each time point, to obtain the original data set of the concrete structure to be inspected, including: S11: Using an integrated camera, high-resolution color or grayscale images of a selected concrete structure are acquired from different perspectives at different times to obtain a multi-view two-dimensional digital image sequence. S12: Synchronously acquire a three-dimensional laser point cloud sequence using a LiDAR scanning device, wherein the three-dimensional laser point cloud sequence is a dense spatial point set synchronized with time. S13: The positioning and navigation system acquires the sensor spatial pose information corresponding to each frame of the multi-view two-dimensional digital image sequence and the three-dimensional laser point cloud sequence. The sensor spatial pose information is used to align the coordinate system of the image and laser point with the global reference coordinate system.

[0008] Preferably, in step S2, fractal dimension analysis is performed on the data of the original dataset of the concrete structure to be detected at each time step to obtain a pixel-level fractal dimension matrix at each time step. The fractal dimension analysis includes, based on an improved box-counting algorithm, introducing orientation sensitivity and a local adaptive window mechanism, calculating the texture complexity of the neighborhood of each pixel in the image, and simultaneously counting the minimum number of boxes containing crack pixels under different scales and orientations of grid coverage. The pixel-level fractal dimension matrix is ​​used as a cue information embedded in the feature layer of the image segmentation model, including: S21: Using each pixel of a frame in a two-dimensional digital image sequence as the center, a variable-size neighborhood window is formed based on an adaptive window mechanism to obtain grids of different scales; S22: Cover a frame of image with the grids of different scales, and for each direction of the image, calculate the texture complexity of the neighborhood of each pixel in the image, and at the same time count the minimum number of boxes containing crack pixels to obtain the multi-scale and orientation-sensitive texture complexity fractal dimension. S23: Organize the texture complexity fractal dimension corresponding to all pixels into a pixel-level fractal dimension matrix; S24: Repeat the above steps for the image at each time step to obtain the pixel-level fractal dimension matrix at each time step.

[0009] Preferably, step S3 involves fusing the pixel-level fractal dimension matrix at each time step with the corresponding images from the multi-view two-dimensional digital image sequence and inputting this into a pre-trained crack segmentation model for crack segmentation, outputting a crack mask image at each time step, including: S31: The pixel-level fractal dimension matrix at each time step is used as the second channel, and the image in the corresponding multi-view two-dimensional digital image sequence is used as the first channel to input the encoder of the pre-trained crack segmentation model for fusion to obtain a fused feature map. The fusion uses the pixel-level fractal dimension matrix at each time step as virtual structure prompt information. S32: Input the fused feature map into the decoder of the pre-trained crack segmentation model for pixel classification, segment the crack, and obtain the crack mask image at each time step. The crack mask image marks the crack pixels as foreground and other pixels as background.

[0010] Preferably, step S4 involves multi-source data fusion and 3D reconstruction of the crack mask image and the corresponding 3D laser point cloud at each time step to obtain a set of crack points projected and marked in the 3D laser point cloud. The multi-source data fusion and 3D reconstruction includes spatial registration of the processing results of all image frames to form a 3D point cloud model containing complete crack information, including: S41: For each moment, the pixels marked as cracks in the crack mask image are used to calculate the corresponding spatial rays based on the sensor spatial pose information. S42: Search for points in the corresponding three-dimensional laser point cloud that fall within the neighborhood of the spatial ray and whose depth error is within a preset threshold range, and mark these points as crack points. Merge all crack points to obtain a crack point set. S43: Spatial registration and point cloud stitching of the crack point set across all image frames are performed, and a filtering and denoising algorithm is applied to eliminate false crack points caused by sensor noise, resulting in a 3D point cloud model containing complete crack information.

[0011] Preferably, in step S5, based on a 3D point cloud model, clustering and principal axis fitting calculations are performed on the set of crack points to obtain the crack spatial trajectory and crack width. The clustering and principal axis fitting calculations include estimating geometric parameters such as crack depth to form a 3D crack model at each moment. The crack width is obtained by calculating the maximum distance between crack points in the direction perpendicular to the crack spatial trajectory, including: S51: Based on the three-dimensional point cloud model, adjacent crack points are compared according to a preset spatial distance threshold, clustering distance is determined, and then they are divided into different crack groups. Each crack group corresponds to a crack unit of a solid structure. S52: The crack grouping is further used for skeleton line extraction to form a spatial skeleton virtual structure for each crack. The skeleton line extraction includes applying a three-dimensional thinning algorithm to extract a coherent main axis from a set of discrete points. S53: Calculate the crack geometric parameters based on the spatial skeleton virtual structure of each crack. The crack geometric parameters include the crack width index, depth index, and orientation angle index. The width index is the maximum lateral distance calculated in the normal direction for sampling points on the skeleton line in a local cross-sectional plane perpendicular to the skeleton direction. S54: Merge the spatial skeleton virtual structure and crack geometry parameters of each crack to obtain the three-dimensional crack model at each moment.

[0012] Preferably, in step S6, multi-temporal crack evolution analysis is performed on the three-dimensional crack model at each time step to quantify temporal characteristics, and finally a crack evolution trajectory model is constructed. The temporal characteristics include crack length growth, crack width variation, and crack fractal dimension variation, including: S61: Register all models to a unified coordinate system; S62: Fix the positioning mark of the three-dimensional crack model at the initial moment; S63: Based on fixed positioning markers, the three-dimensional crack models acquired at different times are automatically aligned using corresponding point matching and spatial proximity methods. Then, the changes in the endpoint position, curve length and shape of each crack at different times are calculated to obtain the time series record of each crack. S64: Based on the time series records of each crack, multi-temporal crack evolution analysis is performed to obtain temporal characteristics; S65: Construct a crack evolution trajectory model based on temporal features.

[0013] On the other hand, an early warning device for cracks in concrete structures is provided, which is applied to a method for early warning of cracks in concrete structures. The device includes: Data acquisition module: used to acquire multi-view two-dimensional digital image sequences of the concrete structure to be inspected at different time points, the corresponding three-dimensional laser point cloud, and the sensor spatial pose information recorded at each time point simultaneously, so as to obtain the original data set of the concrete structure to be inspected; Fractal module: used to perform fractal dimension analysis on the data of the original dataset of the concrete structure to be detected at each time step, and obtain the pixel-level fractal dimension matrix at each time step. The fractal dimension analysis includes: based on the improved box counting algorithm, introducing orientation sensitivity and local adaptive window mechanism, calculating the texture complexity of the neighborhood of each pixel in the image, and simultaneously counting the minimum number of boxes containing crack pixels under different scales and orientation grid coverage. The pixel-level fractal dimension matrix is ​​used as a prompt information to be embedded in the feature layer of the image segmentation model. Crack segmentation module: This module is used to fuse the pixel-level fractal dimension matrix at each time step with the corresponding image from the multi-view two-dimensional digital image sequence and input it into the pre-trained crack segmentation model to segment cracks, outputting a crack mask image at each time step. Crack masking module: used to perform multi-source data fusion and 3D reconstruction of crack mask images and corresponding 3D laser point clouds at each time moment, to obtain a set of crack points projected and marked in the 3D laser point cloud. The multi-source data fusion and 3D reconstruction includes spatial registration of the processing results of all image frames, thereby forming a 3D point cloud model containing complete crack information. Crack clustering module: Based on a 3D point cloud model, it performs clustering and main axis fitting calculations on the set of crack points to obtain the crack spatial trajectory and crack width. The clustering and main axis fitting calculations include estimating geometric parameters such as crack depth to form a 3D crack model at each time step. The crack width is obtained by calculating the maximum distance between crack points in the vertical direction of the crack spatial trajectory. Crack Evolution Module: This module is used to perform multi-temporal crack evolution analysis on the three-dimensional crack model at each time step to quantify temporal characteristics, and finally construct a crack evolution trajectory model. The temporal characteristics include crack length growth, crack width variation, and crack fractal dimension variation. Early warning generation module: used to input the time series features into a pre-trained time series model to obtain predicted values ​​of crack length and crack width, and input the predicted values ​​of crack length and crack width into a prediction result evaluation module to obtain early warning information. The prediction result evaluation module includes evaluating the structural safety status based on the prediction results and providing data support for subsequent maintenance decisions. The pre-trained time series model is constructed based on a long short-term memory neural network.

[0014] On the other hand, a pre-warning device for cracks in concrete structures is provided, the pre-warning device for cracks in concrete structures comprising: a processor; and a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, any one of the pre-warning methods for cracks in concrete structures described above is implemented.

[0015] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for early warning of cracks in concrete structures.

[0016] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: It achieves accurate crack detection under zero-sample conditions, avoiding the arduous task of manual annotation. It can reconstruct the true shape and size of cracks in three-dimensional space, obtain key parameters such as crack width and orientation, automatically record and analyze the propagation trajectory of the same crack, and use time series models to predict future development trends. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of an early warning method for cracks in concrete structures provided by an embodiment of the present invention; Figure 2 This is a flowchart of a crack mask image generation method provided in an embodiment of the present invention; Figure 3 This is a block diagram of an early warning device for cracks in a concrete structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an early warning device for cracks in a concrete structure provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0020] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0021] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0022] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0023] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0024] This invention provides a method for early warning of cracks in concrete structures. This method can be implemented by an early warning device for concrete structure cracks, which can be a terminal or a server. Figure 1 The flowchart shown is for an early warning method for cracks in concrete structures. The processing flow of this method may include the following steps:

[0025] S1: Acquire multi-view two-dimensional digital image sequences of the concrete structure to be inspected at different time points, the corresponding three-dimensional laser point cloud, and the sensor spatial pose information recorded at each time point simultaneously to obtain the original data set of the concrete structure to be inspected. Preferably, S1 includes: S11: Using an integrated camera, high-resolution color or grayscale images of a selected concrete structure are acquired from different perspectives at different times to obtain a multi-view two-dimensional digital image sequence. S12: Synchronously acquire a three-dimensional laser point cloud sequence using a LiDAR scanning device, wherein the three-dimensional laser point cloud sequence is a dense spatial point set synchronized with time. S13: The positioning and navigation system acquires the sensor spatial pose information corresponding to each frame of the multi-view two-dimensional digital image sequence and the three-dimensional laser point cloud sequence. The sensor spatial pose information is used to align the coordinate system of the image and laser point with the global reference coordinate system.

[0026] This embodiment focuses on a crack on the surface of a concrete bridge pier. The method of this invention was applied to conduct multiple inspections over a period of six months, achieving three-dimensional reconstruction and evolution monitoring of the crack. The bridge pier is approximately 5 meters high, and small cracks exist on its surface; initial manual inspection revealed a maximum width of approximately 2 millimeters.

[0027] It should be noted that during the initial inspection (denoted as ), a device integrating an RGB camera and a 3D laser scanner was moved along the surface of the bridge pier to perform a 360-degree scan around the pier. During the scan, the spatial pose of the camera and the corresponding point cloud data were recorded simultaneously for each image captured. By stitching together the frame-by-frame point clouds using a SLAM algorithm, a global 3D point cloud model of the bridge pier surface was constructed, and the pose matrix of each frame relative to this model was obtained. A total of 36 high-resolution images (one image every 10° around the pier's circumference) and corresponding point clouds were acquired on the bridge pier surface. Six months later (denoted as ), the same bridge pier was scanned a second time, acquiring multiple images and point cloud data from the same viewpoint and location. Several reflective markers were pre-attached to the bridge pier during the scan to ensure accurate alignment of the point cloud coordinate systems in both scans.

[0028] S2: Perform fractal dimension analysis on the data of the original dataset of the concrete structure to be detected at each time step to obtain the pixel-level fractal dimension matrix at each time step. The fractal dimension analysis includes: based on the improved box counting algorithm, introducing orientation sensitivity and local adaptive window mechanism, calculating the texture complexity of the neighborhood of each pixel in the image, and simultaneously counting the minimum number of boxes containing crack pixels under different scales and orientations of grid coverage. The pixel-level fractal dimension matrix is ​​used as a prompt information to be embedded in the feature layer of the image segmentation model. Preferably, S2 includes: S21: Using each pixel of a frame in a two-dimensional digital image sequence as the center, a variable-size neighborhood window is formed based on an adaptive window mechanism to obtain grids of different scales; S22: Cover a frame of image with the grids of different scales, and for each direction of the image, calculate the texture complexity of the neighborhood of each pixel in the image, and at the same time count the minimum number of boxes containing crack pixels to obtain the multi-scale and orientation-sensitive texture complexity fractal dimension. S23: Organize the texture complexity fractal dimension corresponding to all pixels into a pixel-level fractal dimension matrix; S24: Repeat the above steps for the image at each time step to obtain the pixel-level fractal dimension matrix at each time step.

[0029] In some embodiments, the fractal dimension matrix is ​​generated using an improved box-counting algorithm to characterize the texture roughness of the crack region; a larger value indicates a more complex crack structure. Its fractal dimension is approximately represented by the following formula. ,in For a side length of Minimum number of cover boxes, ≠ .

[0030] It should be noted that fractal analysis is introduced to uncover the texture features of cracks without relying on pre-labeled crack samples. Specifically, local window scanning is performed on the grayscale image to calculate the fractal dimension of each region. Fractal dimension is a measure of shape complexity; for example, cracks, with their irregular, tortuous lines and high surface roughness, will have a larger fractal dimension relative to the background.

[0031] It should be further explained that the box-counting algorithm is used to approximate the fractal dimension: images covered by grids of different sizes are selected, and the number of grids containing crack pixels at each scale is counted. Preferably, this fractal dimension matrix is ​​provided as a cue to the subsequent semantic segmentation model, aiming to guide the model to focus on pixel regions with high fractal dimensions, thereby achieving zero-shot recognition of cracks. Since the fractal dimension characterizes the complexity of the texture, even if the model has not been trained on specific crack samples, it can effectively distinguish cracks from the background through the cue.

[0032] It should be further explained that in this embodiment, each image of the bridge pier surface acquired at any time is input into the fractal analysis module. For example, an image containing a clear vertical crack is selected for illustration. The module uses a sliding window of pixels to calculate the local fractal dimension pixel by pixel. For the sub-image covered by the window, the Canny operator is first applied to the grayscale image to extract the crack edge pixels to obtain the binary crack morphology. Then, the box counting method is used to calculate its fractal dimension: starting from the binary edge map of the region (canny or other edge detection methods can be used to initially obtain the crack edge pixels), the number of grids required to fill the crack is approximately calculated using grids of different scales. In this embodiment, pixels 1 and 2 are taken, and the corresponding values ​​are calculated and 3 are substituted into the fractal dimension calculation formula above to obtain the fractal dimension of the region. Then, the window is slid to the next position of the image and the calculation is repeated until the entire image is covered. Finally, a fractal dimension matrix of the same size as the image is output. In this matrix, the fractal dimension of the concrete background area of ​​the bridge pier is generally low, while the vertical thin line area where the cracks are located exhibits a significantly higher fractal dimension (above 1.5, while the background is around 1.2) due to its irregular and tortuous edges. Thus, the matrix effectively forms a "heatmap" highlighting the location of cracks: pixels with higher values ​​are more likely to belong to cracks. Visualizing this matrix by mapping pixel values ​​to grayscale or color images clearly shows that crack locations are clearly highlighted. After normalizing the matrix, the fractal analysis module inputs it as prompt information into the semantic segmentation module.

[0033] S3: The pixel-level fractal dimension matrix at each time step and the corresponding image in the multi-view two-dimensional digital image sequence are fused and input into the pre-trained crack segmentation model to perform crack segmentation, and the crack mask image at each time step is output. Preferably, such as Figure 2As shown, S3 includes: S31: The pixel-level fractal dimension matrix at each time step is used as the second channel, and the image in the corresponding multi-view two-dimensional digital image sequence is used as the first channel to input the encoder of the pre-trained crack segmentation model for fusion to obtain a fused feature map. The fusion uses the pixel-level fractal dimension matrix at each time step as virtual structure prompt information. S32: Input the fused feature map into the decoder of the pre-trained crack segmentation model for pixel classification, segment the crack, and obtain the crack mask image at each time step. The crack mask image marks the crack pixels as foreground and other pixels as background.

[0034] In some embodiments, the pixel-level fractal dimension matrix serves as a pre-trained crack segmentation model that supports cue input. The fractal dimension matrix is ​​fused and encoded with the image to be segmented and then input into the model, enabling the model to perform zero-shot segmentation on crack targets that have not been trained on during decoding and inference. By guiding the model to focus on pixel regions with high fractal dimensions through cue input, the model adaptively improves its sensitivity to crack edges and small cracks, thereby improving the recall rate of crack detection.

[0035] It should be noted that a large-scale pre-trained segmentation model based on the Transformer architecture was chosen as the foundation. This model has the ability to segment objects of any new category and provides an interface for prompting input.

[0036] It should be further explained that in this embodiment, a general segmentation network based on VisionTransformer is selected as the base model, which has an interface for accepting additional cue signals. First, the fractal dimension matrix F obtained in the previous step is used as the second channel, and together with the original image (the first channel), they form a dual-channel input to the model's encoder. The model extracts features from the image with the cues, and outputs a binary segmentation mask of the same size as the input during the decoding stage. Since the model has not seen specific crack samples during training, the provided fractal cues effectively guide the model to focus on highly irregular texture regions. Ultimately, the model successfully distinguishes the crack pixels in the bridge pier image from the background. The semantic segmentation module outputs a crack mask image, where crack pixels are marked as white connected regions, and other regions are black. Manual comparison and verification show that the crack regions in the mask match the visible cracks in the original image well, indicating that zero-shot segmentation achieves high reliability and accuracy.

[0037] S4: Perform multi-source data fusion and 3D reconstruction on the crack mask image and the corresponding 3D laser point cloud at each time moment to obtain a set of crack points that are projected, mapped and marked in the 3D laser point cloud. The multi-source data fusion and 3D reconstruction includes spatial registration of the processing results of all image frames to form a 3D point cloud model containing complete crack information. Preferably, S4 includes: S41: For each moment, the pixels marked as cracks in the crack mask image are used to calculate the corresponding spatial rays based on the sensor spatial pose information. S42: Search for points in the corresponding three-dimensional laser point cloud that fall within the neighborhood of the spatial ray and whose depth error is within a preset threshold range, and mark these points as crack points. Merge all crack points to obtain a crack point set. S43: Spatial registration and point cloud stitching of the crack point set across all image frames are performed, and a filtering and denoising algorithm is applied to eliminate false crack points caused by sensor noise, resulting in a 3D point cloud model containing complete crack information.

[0038] In some embodiments, using pre-calibrated camera intrinsic and extrinsic parameters, crack pixels in the image are mapped to corresponding positions in the 3D point cloud via ray projection: when a pixel is segmented into a crack, a point in the point cloud where its line of sight intersects and is at the same depth from the camera is identified and marked as a crack point. By traversing all crack pixels in the mask image and projecting them into the point cloud for matching, the image crack information is fused to the 3D point cloud. During the registration process, distance threshold constraints and neighborhood coherence checks are set to filter out noisy matching points, accurately fusing the crack mask into the point cloud model.

[0039] It should be noted that this step overcomes the dependence of traditional supervised segmentation on the dataset, and achieves robust detection of cracks with different structures and morphologies under zero-shot conditions, greatly improving its versatility.

[0040] It should be further explained that in the spatial registration illustration, the image coordinate system and the point cloud coordinate system are first transformed and unified using the camera's external parameters (pose matrix and translation vector). Specifically, for any 3D point in the point cloud, its projected position on the image can be calculated based on the camera pose, i.e., through a projection model.

[0041] Preferably, the camera pose corresponding to the image time is read. For each crack foreground pixel in the mask image, the distance to the point is analyzed to see if it is consistent with the depth of the camera's line of sight (e.g., there is an echo on the depth map). If so, the point is identified as a crack point. Since multiple frames of images cover different parts of the pier, the system will generate a portion of the crack point cloud for each image. The results from all images are transformed to a unified coordinate system and accumulated and fused to obtain a complete crack point cloud model of the pier. During the fusion process, the following constraints are introduced to reduce the risk of mismatch: the maximum distance of the ray search is limited, and pixels exceeding the effective range are ignored; the projection results of adjacent pixels are checked for coherence to avoid isolated noise points. The crack point cloud obtained after fusion may still contain a small number of scattered noise points. In this embodiment, a statistical filtering method is used to remove local outliers in the point cloud to ensure that the crack point cloud accurately depicts the location of the actual crack. After processing, the cracks on the pier surface are clearly marked in the point cloud model, and each crack point carries its spatial coordinate attributes. Points in the crack area are marked in red to distinguish them from the background point cloud.

[0042] S5: Based on the three-dimensional point cloud model, clustering and main axis fitting calculations are performed on the set of crack points to obtain the crack spatial trajectory and crack width. The clustering and main axis fitting calculations include estimating geometric parameters such as crack depth to form a three-dimensional crack model at each moment. The crack width is obtained by calculating the maximum distance between crack points in the vertical direction of the crack spatial trajectory. Preferably, S5 includes: S51: Based on the three-dimensional point cloud model, adjacent crack points are compared according to a preset spatial distance threshold, clustering distance is determined, and then they are divided into different crack groups. Each crack group corresponds to a crack unit of a solid structure. S52: The crack grouping is further used for skeleton line extraction to form a spatial skeleton virtual structure for each crack. The skeleton line extraction includes applying a three-dimensional thinning algorithm to extract a coherent main axis from a set of discrete points. S53: Calculate the crack geometric parameters based on the spatial skeleton virtual structure of each crack. The crack geometric parameters include the crack width index, depth index, and orientation angle index. The width index is the maximum lateral distance calculated in the normal direction for sampling points on the skeleton line in a local cross-sectional plane perpendicular to the skeleton direction. S54: Merge the spatial skeleton virtual structure and crack geometry parameters of each crack to obtain the three-dimensional crack model at each moment.

[0043] In some embodiments, when a structural surface reference plane is available, the crack depth can also be estimated by the degree of vertical indentation of the crack point relative to the surface.

[0044] It should be noted that in obtaining After obtaining the crack point cloud at a given time, the crack feature modeling module analyzes and processes it. First, the points are divided into several subsets based on the distance between them, with each subset representing an independent crack. The criterion is: if the Euclidean distance between two points is less than a preset threshold (e.g., 5cm), they are considered to belong to the same crack cluster; otherwise, they belong to different cracks. Assuming this embodiment identifies two main cracks, the longer one is selected for further processing. Spatial curve fitting is performed on the crack point set to extract the crack skeleton line. Specifically, the covariance matrix of the point set is first calculated, and the direction of the first principal component is obtained to get an initial estimate of the overall crack direction; then, guided by this direction, a 3D thinning algorithm is used to extract the continuous central axis running through the entire point set. For example, a stepwise shrinking algorithm can be used to "skeletonize" the point cloud, or the point cloud can be projected along the main direction and fitted with multiple polylines to approximate the crack path. The resulting skeleton line consists of several ordered points, depicting the spatial trajectory and tortuous extension of the crack on the structural surface. Next, the crack width is calculated. For several sampling points selected along the skeleton line, the maximum lateral distance of the crack point cloud in the plane perpendicular to the skeleton (i.e., the local normal plane) is calculated. The maximum value of these maximum distances approximately represents the maximum width of the crack, while the average width value of all sampling points is used as the average width. In this embodiment, the maximum width of the crack is calculated to be approximately 2.0 mm, and the average width is approximately 0.5 mm. If it is necessary to estimate the crack depth, the crack points can be compared with the uncracked concrete surface: since the point cloud only covers the surface in this example, the depth can be approximately determined by analyzing the normal direction of the crack point cloud (if the crack is accompanied by spalling to form a groove, the crack point may be lower than the surrounding surface points in the normal direction). After the above processing, a three-dimensional model of the crack is established, including: the crack skeleton line point sequence and its spatial coordinates, crack length (obtained by accumulating the skeleton curves, with a total length of approximately 2.3 m), maximum width (2.0 mm), average width (0.5 mm), and overall crack orientation (the skeleton's projection on the horizontal plane is nearly vertical), etc. Similarly, the other shorter crack was processed in a similar way to obtain a three-dimensional model with a length of approximately 0.8m, an average width of 0.3mm, and a nearly horizontal orientation. At this point, the system obtained complete three-dimensional information about the pier crack at any given time.

[0045] S6: Perform multi-temporal crack evolution analysis on the three-dimensional crack model at each time point to quantify temporal characteristics, and finally construct a crack evolution trajectory model. The temporal characteristics include crack length growth, crack width variation, and crack fractal dimension variation. Preferably, S6 includes: S61: Register all models to a unified coordinate system; S62: Fix the positioning mark of the three-dimensional crack model at the initial moment; S63: Based on fixed positioning markers, the three-dimensional crack models acquired at different times are automatically aligned using corresponding point matching and spatial proximity methods. Then, the changes in the endpoint position, curve length and shape of each crack at different times are calculated to obtain the time series record of each crack. S64: Based on the time series records of each crack, multi-temporal crack evolution analysis is performed to obtain temporal characteristics; S65: Construct a crack evolution trajectory model based on temporal features.

[0046] In some embodiments, the three-dimensional model of the crack is registered to a unified coordinate system by feature alignment or marker matching in order to track the geometric changes of the same crack over time and to statistically analyze the trend of the crack fractal dimension over time.

[0047] It should be noted that the crack point cloud is grouped according to the crack it belongs to through connectivity analysis: if two points are spatially adjacent (less than a preset threshold), they are considered to belong to the same physical crack. This allows the global point set to be divided into one or more cracks. If reference plane data of the structural surface is available, the crack depth (e.g., the maximum distance a crack point can indent relative to the surface) can also be estimated as another indicator of severity.

[0048] It should be further explained that, approximately six months later, the same bridge pier underwent the same inspection process again, yielding 3D crack model data at time [time]. Because the same equipment and calibration were used, and fixed markers were placed on the bridge pier during the scanning process for coordinate alignment, the point cloud model and the [model name] could be seamlessly registered and compared in the same coordinate system. After loading the crack models from [time] and [time], the crack evolution analysis module first precisely aligned the two point clouds by identifying common reference feature points (such as edges on the bridge pier or pre-embedded reflective markers), ensuring the coordinate system consistency of the two scans. Then, the changes in the same crack at the two time points were compared. For example, for the main vertical crack processed above, the starting points of the [model name] and [model name] skeleton lines were aligned, and the difference in vertical distance between the two skeleton endpoints was calculated. It was found that at time [time], the skeleton extended downwards by approximately 0.4m, meaning the crack length increased from 2.3m to 2.7m, an increase of approximately 17%. Furthermore, comparing the width, the maximum width was 2.0mm at time [time], increasing to 3.5mm at time [time], an increase of 1.5mm, indicating that the crack significantly widened during this period. For another short horizontal crack, a slight increase in length (approximately 0.1 m) was observed, while the width remained relatively unchanged. Summarizing these data creates a list of information regarding the crack's evolution trajectory. For example, for the first crack, the lengths (m and m) and maximum widths (mm and mm) were recorded at the specified times. This allows for calculation of an increase in length (m) and width (mm) over a period of six months, with an average monthly expansion rate of approximately 0.067 m / month (length) and 0.25 mm / month (width). These indicators clearly quantify the crack's expansion rate and reflect its significant expansion over the six-month period.

[0049] S7: Input the time series features into the pre-trained time series model to obtain the predicted values ​​of crack length and crack width. Input the predicted values ​​of crack length and crack width into the prediction result evaluation module to obtain early warning information. The prediction result evaluation module includes evaluating the structural safety status based on the prediction results and providing data support for subsequent maintenance decisions. The pre-trained time series model is constructed based on a long short-term memory neural network.

[0050] In some embodiments, the crack evolution trajectory is modeled using the following metrics: displacement change of the skeleton point position, rate of change of crack length, and overlap metric of crack boundary morphology, wherein the overlap metric of boundary morphology is defined as: ,in Indicates the crack at time The area For adjacent monitoring periods, this indicator is used to quantify the morphological similarity of crack regions in different time phases.

[0051] It should be noted that by analyzing the changes in multi-temporal crack feature sequences, a predictive model of crack parameters over time can be established. In simple cases, it can be assumed that the crack length increases linearly with time, and the approximate growth rate can be estimated using the results of the two most recent measurements. In more complex cases, a time-series prediction model using machine learning is introduced to improve accuracy. For example, this invention preferably uses a Long Short-Term Memory (LSTM) neural network to model the time-series features of crack expansion. LSTM is a special type of recurrent neural network that controls the memory and forgetting of information through internal mechanisms such as "forget gate," "input gate," and "output gate." Compared to traditional linear models or simple RNNs, LSTM is better at capturing long-term dependencies and can more accurately describe the nonlinear trend of crack expansion. In specific implementation, the sequence data of crack length, width, etc., obtained from historical monitoring are used as training samples and input into the LSTM model for learning. A sequence length of a certain time window is selected as the input step size, and the model is trained to predict the crack parameter values ​​at the next time step. During training, a portion of the historical data is used as the training set, and the remainder is used as the validation / test set. The prediction error is converged by iteratively optimizing the model parameters. The loss function can be measured by the mean squared error (MSE), and the evaluation metric can be measured by the mean absolute percentage error (MAPE) to measure the prediction accuracy.

[0052] It should be further explained that by using the trained LSTM model to infer the current crack feature sequence, predicted values ​​such as crack length and width can be obtained for a future period of time. For example, based on data from multiple monitoring sessions over the previous six months, the model predicts that after another six months, the crack length may increase by approximately 0.4m, reaching about 3.1m, and the maximum width may approach 5mm. If the prediction results indicate that a crack size will exceed a safety threshold in the future, the system can issue an early warning, prompting maintenance personnel to take timely intervention measures. It should be noted that this invention does not limit the specific prediction algorithm; any model based on time-series data (such as ARIMA, autoregressive networks, etc.) can replace LSTM for trend prediction as needed to improve accuracy and robustness. Once the future crack development is predicted, such as the crack length being expected to exceed a predetermined threshold after a certain period of time, an early warning can be issued, shifting from passive monitoring to proactive prevention, thus improving the foresight and safety of structural operation and maintenance.

[0053] Preferably, based on the limited data from the two periods mentioned above, the trend assessment unit of the crack evolution analysis module can attempt to predict future crack conditions. In practical applications, monitoring frequency is usually higher (e.g., once per quarter or monthly) to obtain sufficient time-series data for training the prediction model. In this embodiment, due to the limited data from only two periods, it is difficult to train a complex model; therefore, it is temporarily assumed that crack extension follows a linear trend: based on this estimate, after another 6 months (i.e., at that time), the crack length may increase by approximately 0.4m, reaching approximately 3.1m, and the maximum width may approach 5mm. Of course, the actual growth may be non-linear due to environmental factors. To improve prediction accuracy, more historical data can be introduced into future monitoring, and more advanced time-series modeling algorithms can be used, such as using an LSTM model for training and prediction under the condition of obtaining monthly data, or using an ARIMA model combined with historical trends for extrapolation. With richer time-series data, the method of this embodiment can be used to train an LSTM model on crack extension to obtain more refined trend prediction results. Once a prediction indicates that the crack width may exceed the safety threshold within a year, the system will immediately issue an early warning, reminding the maintenance department to plan repair measures in advance to prevent the crack from further developing and endangering structural safety.

[0054] This embodiment demonstrates that the present invention works effectively in real-world engineering scenarios. Without requiring specialized crack training, it successfully detects minute cracks that are difficult to detect with the naked eye in complex backgrounds and accurately maps them onto a 3D model of the structural surface. Simultaneously, using data from two monitoring sessions, the system quantifies the crack propagation rate and trend, providing a scientific basis for subsequent structural health management. Compared to traditional manual inspections and 2D image analysis methods, this invention provides more comprehensive and quantitative results, significantly improving the intelligence level of crack monitoring and management.

[0055] The above is an introduction to the method embodiments. The following describes the solution described in this application through device embodiments.

[0056] Figure 3 This is a block diagram illustrating an early warning device for cracks in a concrete structure according to an exemplary embodiment. The device is used in a method for early warning of cracks in a concrete structure. (Refer to...) Figure 3 The device includes a data acquisition module, a fractal module, a crack segmentation module, a crack masking module, a crack clustering module, a crack evolution module, and an early warning generation module.

[0057] Data acquisition module: used to acquire multi-view two-dimensional digital image sequences of the concrete structure to be inspected at different time points, the corresponding three-dimensional laser point cloud, and the sensor spatial pose information recorded at each time point simultaneously, so as to obtain the original data set of the concrete structure to be inspected; Fractal module: used to perform fractal dimension analysis on the data of the original dataset of the concrete structure to be detected at each time step, and obtain the pixel-level fractal dimension matrix at each time step. The fractal dimension analysis includes: based on the improved box counting algorithm, introducing orientation sensitivity and local adaptive window mechanism, calculating the texture complexity of the neighborhood of each pixel in the image, and simultaneously counting the minimum number of boxes containing crack pixels under different scales and orientation grid coverage. The pixel-level fractal dimension matrix is ​​used as a prompt information to be embedded in the feature layer of the image segmentation model. Crack segmentation module: This module is used to fuse the pixel-level fractal dimension matrix at each time step with the corresponding image from the multi-view two-dimensional digital image sequence and input it into the pre-trained crack segmentation model to segment cracks, outputting a crack mask image at each time step. Crack masking module: used to perform multi-source data fusion and 3D reconstruction of crack mask images and corresponding 3D laser point clouds at each time moment, to obtain a set of crack points projected and marked in the 3D laser point cloud. The multi-source data fusion and 3D reconstruction includes spatial registration of the processing results of all image frames, thereby forming a 3D point cloud model containing complete crack information. Crack clustering module: Based on a 3D point cloud model, it performs clustering and main axis fitting calculations on the set of crack points to obtain the crack spatial trajectory and crack width. The clustering and main axis fitting calculations include estimating geometric parameters such as crack depth to form a 3D crack model at each time step. The crack width is obtained by calculating the maximum distance between crack points in the vertical direction of the crack spatial trajectory. Crack Evolution Module: This module is used to perform multi-temporal crack evolution analysis on the three-dimensional crack model at each time step to quantify temporal characteristics, and finally construct a crack evolution trajectory model. The temporal characteristics include crack length growth, crack width variation, and crack fractal dimension variation. Early warning generation module: used to input the time series features into a pre-trained time series model to obtain predicted values ​​of crack length and crack width, and input the predicted values ​​of crack length and crack width into a prediction result evaluation module to obtain early warning information. The prediction result evaluation module includes evaluating the structural safety status based on the prediction results and providing data support for subsequent maintenance decisions. The pre-trained time series model is constructed based on a long short-term memory neural network.

[0058] A device for early warning of cracks in concrete structures, comprising: a processor; and a memory storing computer-readable instructions, wherein when executed by the processor, the computer-readable instructions implement the method described in any one of the above-described methods for early warning of cracks in concrete structures.

[0059] A computer-readable storage medium, characterized in that the computer-readable storage medium stores program code, the program code being invoked by a processor to execute the method as described in any one of claims 1 to 7.

[0060] Figure 4 This is a structural schematic diagram of an early warning device for cracks in a concrete structure provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the early warning device for cracks in concrete structures may include the above-mentioned Figure 3 The illustrated early warning device for cracks in concrete structures. Optionally, the early warning device 410 for cracks in concrete structures may include a first processor 2001.

[0061] Optionally, the early warning device 410 for cracks in concrete structures may also include a memory 2002 and a transceiver 2003.

[0062] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0063] The following is combined Figure 4 A detailed introduction to each component of the early warning device 410 for cracks in concrete structures: The first processor 2001 is the control center of the early warning device 410 for concrete structure cracks. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0064] Optionally, the first processor 2001 can perform various functions of the early warning device 410 for concrete structure cracks by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0065] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0066] In a specific implementation, as one example, the early warning device 410 for cracks in concrete structures may also include multiple processors, for example... Figure 4 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0067] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0068] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the concrete structure crack early warning device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0069] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0070] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0071] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected to the interface circuit of the concrete structure crack early warning device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0072] It should be noted that, Figure 4 The structure of the early warning device 410 for concrete structure cracks shown in the diagram does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0073] Furthermore, the technical effect of the early warning device 410 for concrete structure cracks can be referred to the technical effect of the early warning method for concrete structure cracks described in the above method embodiments, and will not be repeated here.

[0074] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0075] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0076] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0077] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0078] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0079] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0080] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0081] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0082] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0084] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0085] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for early warning of cracks in concrete structures, characterized in that, The method includes: S1: Acquire multi-view two-dimensional digital image sequences of the concrete structure to be inspected at different time points, the corresponding three-dimensional laser point cloud, and the sensor spatial pose information recorded at each time point simultaneously to obtain the original data set of the concrete structure to be inspected. S2: Perform fractal dimension analysis on the data of the original dataset of the concrete structure to be detected at each time step to obtain the pixel-level fractal dimension matrix at each time step. The fractal dimension analysis includes: based on the improved box counting algorithm, introducing orientation sensitivity and local adaptive window mechanism, calculating the texture complexity of the neighborhood of each pixel in the image, and simultaneously counting the minimum number of boxes containing crack pixels under different scales and orientations of grid coverage. The pixel-level fractal dimension matrix is ​​used as a prompt information to be embedded in the feature layer of the image segmentation model. S3: The pixel-level fractal dimension matrix at each time step and the corresponding image in the multi-view two-dimensional digital image sequence are fused and input into the pre-trained crack segmentation model to perform crack segmentation, and the crack mask image at each time step is output. S4: Perform multi-source data fusion and 3D reconstruction on the crack mask image and the corresponding 3D laser point cloud at each time moment to obtain a set of crack points that are projected, mapped and marked in the 3D laser point cloud. The multi-source data fusion and 3D reconstruction includes spatial registration of the processing results of all image frames to form a 3D point cloud model containing complete crack information. S5: Based on the three-dimensional point cloud model, clustering and main axis fitting calculations are performed on the set of crack points to obtain the crack spatial trajectory and crack width. The clustering and main axis fitting calculations include estimating geometric parameters such as crack depth to form a three-dimensional crack model at each moment. The crack width is obtained by calculating the maximum distance between crack points in the vertical direction of the crack spatial trajectory. S6: Perform multi-temporal crack evolution analysis on the three-dimensional crack model at each time point to quantify temporal characteristics, and finally construct a crack evolution trajectory model. The temporal characteristics include crack length growth, crack width variation, and crack fractal dimension variation. S7: Input the time series features into the pre-trained time series model to obtain the predicted values ​​of crack length and crack width. Input the predicted values ​​of crack length and crack width into the prediction result evaluation module to obtain early warning information. The prediction result evaluation module includes evaluating the structural safety status based on the prediction results and providing data support for subsequent maintenance decisions. The pre-trained time series model is constructed based on a long short-term memory neural network.

2. The method for early warning of cracks in concrete structures according to claim 1, characterized in that, S1 acquires a multi-view two-dimensional digital image sequence of the concrete structure to be inspected at different time points, the corresponding three-dimensional laser point cloud, and the synchronously recorded sensor spatial pose information at each time point, to obtain the original data set of the concrete structure to be inspected, including: S11: Using an integrated camera, high-resolution color or grayscale images of a selected concrete structure are acquired from different time points and perspectives to obtain a multi-view two-dimensional digital image sequence. S12: Synchronously acquire a three-dimensional laser point cloud sequence using a LiDAR scanning device, wherein the three-dimensional laser point cloud sequence is a dense spatial point set synchronized with time. S13: The positioning and navigation system acquires the sensor spatial pose information corresponding to each frame of the multi-view two-dimensional digital image sequence and the three-dimensional laser point cloud sequence. The sensor spatial pose information is used to align the coordinate system of the image and laser point with the global reference coordinate system.

3. The method for early warning of cracks in concrete structures according to claim 1, characterized in that, S2 performs fractal dimension analysis on the data of the original dataset of the concrete structure to be detected at each time step to obtain a pixel-level fractal dimension matrix at each time step. The fractal dimension analysis includes, based on an improved box counting algorithm, introducing orientation sensitivity and a local adaptive window mechanism, calculating the texture complexity of the neighborhood of each pixel in the image, and simultaneously counting the minimum number of boxes containing crack pixels under different scales and orientations of grid coverage. The pixel-level fractal dimension matrix is ​​used as a cue information embedded in the feature layer of the image segmentation model, including: S21: Using each pixel of a frame in a two-dimensional digital image sequence as the center, a variable-size neighborhood window is formed based on an adaptive window mechanism to obtain grids of different scales; S22: Cover a frame of image with the grids of different scales, and for each direction of the image, calculate the texture complexity of the neighborhood of each pixel in the image, and at the same time count the minimum number of boxes containing crack pixels to obtain the multi-scale and orientation-sensitive texture complexity fractal dimension. S23: Organize the texture complexity fractal dimension corresponding to all pixels into a pixel-level fractal dimension matrix; S24: Repeat the above steps for the image at each time step to obtain the pixel-level fractal dimension matrix at each time step.

4. The method for early warning of cracks in concrete structures according to claim 1, characterized in that, S3 involves fusing the pixel-level fractal dimension matrix at each time step with the corresponding images from the multi-view two-dimensional digital image sequence and inputting this into a pre-trained crack segmentation model for crack segmentation, outputting a crack mask image at each time step, including: S31: The pixel-level fractal dimension matrix at each time step is used as the second channel, and the image in the corresponding multi-view two-dimensional digital image sequence is used as the first channel to input the encoder of the pre-trained crack segmentation model for fusion to obtain a fused feature map. The fusion uses the pixel-level fractal dimension matrix at each time step as virtual structure prompt information. S32: Input the fused feature map into the decoder of the pre-trained crack segmentation model for pixel classification, segment the crack, and obtain the crack mask image at each time step. The crack mask image marks the crack pixels as foreground and other pixels as background.

5. The method for early warning of cracks in concrete structures according to claim 1, characterized in that, S4 involves multi-source data fusion and 3D reconstruction of the crack mask image and corresponding 3D laser point cloud at each time step, resulting in a set of crack points projected and marked in the 3D laser point cloud. The multi-source data fusion and 3D reconstruction includes spatial registration of the processing results of all image frames, thereby forming a 3D point cloud model containing complete crack information, including: S41: For each moment, the pixels marked as cracks in the crack mask image are used to calculate the corresponding spatial rays based on the sensor spatial pose information. S42: Search for points in the corresponding three-dimensional laser point cloud that fall within the neighborhood of the spatial ray and whose depth error is within a preset threshold range, and mark these points as crack points. Merge all crack points to obtain a crack point set. S43: Spatial registration and point cloud stitching of the crack point set across all image frames are performed, and a filtering and denoising algorithm is applied to eliminate false crack points caused by sensor noise, resulting in a 3D point cloud model containing complete crack information.

6. The method for early warning of cracks in concrete structures according to claim 1, characterized in that, The S5 method, based on a 3D point cloud model, performs clustering and principal axis fitting calculations on the set of crack points to obtain the crack spatial trajectory and crack width. The clustering and principal axis fitting calculations include estimating geometric parameters such as crack depth to form a 3D crack model at each time step. The crack width is obtained by calculating the maximum distance between crack points in the direction perpendicular to the crack spatial trajectory, including: S51: Based on the three-dimensional point cloud model, adjacent crack points are compared according to a preset spatial distance threshold, clustering distance is determined, and then they are divided into different crack groups. Each crack group corresponds to a crack unit of a solid structure. S52: The crack grouping is further used for skeleton line extraction to form a spatial skeleton virtual structure for each crack. The skeleton line extraction includes applying a three-dimensional thinning algorithm to extract a coherent main axis from a set of discrete points. S53: Calculate the crack geometric parameters based on the spatial skeleton virtual structure of each crack. The crack geometric parameters include the crack width index, depth index, and orientation angle index. The width index is the maximum lateral distance calculated in the normal direction for sampling points on the skeleton line in a local cross-sectional plane perpendicular to the skeleton direction. S54: Merge the spatial skeleton virtual structure and crack geometry parameters of each crack to obtain the three-dimensional crack model at each moment.

7. The method for early warning of cracks in concrete structures according to claim 1, characterized in that, The S6 method performs multi-temporal crack evolution analysis on the three-dimensional crack model at each time step to quantify temporal characteristics, and finally constructs a crack evolution trajectory model. These temporal characteristics include crack length growth, crack width variation, and crack fractal dimension variation, including: S61: Register all models to a unified coordinate system; S62: Fix the positioning mark of the three-dimensional crack model at the initial moment; S63: Based on fixed positioning markers, the three-dimensional crack models acquired at different times are automatically aligned using corresponding point matching and spatial proximity methods. Then, the changes in the endpoint position, curve length and shape of each crack at different times are calculated to obtain the time series record of each crack. S64: Based on the time series records of each crack, multi-temporal crack evolution analysis is performed to obtain temporal characteristics; S65: Construct a crack evolution trajectory model based on temporal features.

8. A device for early warning of cracks in concrete structures, wherein the device is used to implement the method for early warning of cracks in concrete structures as described in any one of claims 1-7, characterized in that, The device includes: Data acquisition module: used to acquire multi-view two-dimensional digital image sequences of the concrete structure to be inspected at different time points, the corresponding three-dimensional laser point cloud, and the sensor spatial pose information recorded at each time point simultaneously, so as to obtain the original data set of the concrete structure to be inspected; Fractal module: used to perform fractal dimension analysis on the data of the original dataset of the concrete structure to be detected at each time step, and obtain the pixel-level fractal dimension matrix at each time step. The fractal dimension analysis includes: based on the improved box counting algorithm, introducing orientation sensitivity and local adaptive window mechanism, calculating the texture complexity of the neighborhood of each pixel in the image, and simultaneously counting the minimum number of boxes containing crack pixels under different scales and orientation grid coverage. The pixel-level fractal dimension matrix is ​​used as a prompt information to be embedded in the feature layer of the image segmentation model. Crack segmentation module: This module is used to fuse the pixel-level fractal dimension matrix at each time step with the corresponding image from the multi-view two-dimensional digital image sequence and input it into the pre-trained crack segmentation model to segment cracks, outputting a crack mask image at each time step. Crack masking module: used to perform multi-source data fusion and 3D reconstruction of crack mask images and corresponding 3D laser point clouds at each time moment, to obtain a set of crack points projected and marked in the 3D laser point cloud. The multi-source data fusion and 3D reconstruction includes spatial registration of the processing results of all image frames, thereby forming a 3D point cloud model containing complete crack information. Crack clustering module: Based on a 3D point cloud model, it performs clustering and main axis fitting calculations on the set of crack points to obtain the crack spatial trajectory and crack width. The clustering and main axis fitting calculations include estimating geometric parameters such as crack depth to form a 3D crack model at each time step. The crack width is obtained by calculating the maximum distance between crack points in the vertical direction of the crack spatial trajectory. Crack Evolution Module: This module is used to perform multi-temporal crack evolution analysis on the three-dimensional crack model at each time step to quantify temporal characteristics, and finally construct a crack evolution trajectory model. The temporal characteristics include crack length growth, crack width variation, and crack fractal dimension variation. Early warning generation module: used to input the time series features into a pre-trained time series model to obtain predicted values ​​of crack length and crack width, and input the predicted values ​​of crack length and crack width into a prediction result evaluation module to obtain early warning information. The prediction result evaluation module includes evaluating the structural safety status based on the prediction results and providing data support for subsequent maintenance decisions. The pre-trained time series model is constructed based on a long short-term memory neural network.

9. An early warning device for cracks in concrete structures, characterized in that, The early warning processor for cracks in the concrete structure; a memory storing computer-readable instructions, which, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images