A visual identification method for stress anomaly cracks of a bidirectional eccentric compression member

Abstract

This invention relates to the field of general image data processing or generation technology, and discloses a visual recognition method for cracks in bidirectional eccentrically compressed components exhibiting stress anomalies. The method includes: simultaneously acquiring visible light images, long-wave infrared thermal imaging images, and ambient lighting parameters; performing multimodal image spatiotemporal alignment and correlation with lighting parameters; performing feedforward lighting compensation on the visible light images based on a lighting model query matching enhancement operator; fusing the enhanced visible light images and infrared images into a convolutional neural network to segment the cracks using dual-stream heterogeneous features; performing morphological processing and topological analysis on the crack mask to extract geometric parameters; and inferring the stress anomaly type by combining a crack-stress anomaly pattern knowledge base. This invention improves the robustness, accuracy, and engineering diagnostic value of crack recognition through multimodal perception, adaptive lighting compensation, and cross-modal intelligent diagnosis.

Description

Technical Field

[0001] This invention belongs to the field of general image data processing or generation technology, and specifically relates to a method for visually identifying cracks in bidirectional eccentrically compressed components with abnormal stress. Background Technology

[0002] In the field of structural health monitoring in civil engineering, crack identification in concrete members is a crucial step in assessing their stress state and safety performance. As a core component of important load-bearing structures such as high-rise buildings and bridge piers, biaxially eccentrically compressed members often exhibit abnormal stress primarily through the morphology, orientation, and distribution characteristics of surface cracks.

[0003] In recent years, computer vision-based automatic crack detection technology has gradually become the mainstream method for on-site monitoring due to its advantages such as non-contact operation, high efficiency, and remote deployment. This type of method typically relies on image acquisition equipment to obtain images of the component surface, and then extracts and quantifies cracks through steps such as image preprocessing, edge enhancement, threshold segmentation, and morphological analysis.

[0004] The accuracy of crack visual recognition is highly dependent on image quality, especially the clarity and contrast of crack edges. However, in actual construction sites, the ambient lighting conditions are complex and variable—strong light reflection, shadow occlusion, day-night cycle, and weather changes cause drastic fluctuations in imaging brightness, which can easily reduce the grayscale difference between the crack area and the background, making the crack edges appear blurred, broken, or even completely obscured.

[0005] Traditional image processing methods generally employ adaptive thresholding strategies to address uneven illumination. However, these methods exhibit reduced robustness in low-light or high dynamic range scenarios, making it difficult to distinguish weak crack signals from noise interference, leading to a sharp increase in false negative rates. More critically, existing algorithms lack real-time perception and response mechanisms to changes in ambient light, failing to dynamically calibrate enhancement parameters and segmentation thresholds immediately after image acquisition. This results in subsequent recognition processes being based on distorted or information-deficient images.

[0006] Existing technologies therefore face multiple contradictions:

[0007] On the one hand, high-fidelity image input is required for crack recognition accuracy;

[0008] On the other hand, dynamic lighting interference at the scene continuously weakens the image signal-to-noise ratio;

[0009] Meanwhile, traditional preprocessing modules are static or semi-static designs, which cannot integrate multi-dimensional light environment information, nor can they achieve closed-loop feedback adjustment of processing parameters.

[0010] These problems are particularly prominent in typical low-light scenarios such as nighttime construction, tunnel interiors, or rainy weather, severely limiting the reliability and practicality of visual recognition systems in real engineering environments. Therefore, there is an urgent need for a crack visual recognition method that can actively sense the spectral characteristics of the environment, adaptively adjust image enhancement strategies, and calibrate segmentation parameters in real time, in order to overcome the bottleneck of limited recognition performance of existing technologies under complex lighting conditions. Summary of the Invention

[0011] To address the technical problems of existing methods for visually identifying cracks in bidirectional eccentrically compressed components, such as blurred crack edges due to dynamic lighting interference at the construction site, high missed detection rate under low illumination conditions, and inability to perform real-time illumination compensation, this invention provides a method for visually identifying cracks in bidirectional eccentrically compressed components under abnormal stress.

[0012] This invention constructs a closed-loop identification system that integrates multispectral imaging, active ambient light perception and compensation, and stress anomaly inference based on physical prior knowledge. This system enables highly robust and accurate detection and cause diagnosis of component cracks under complex dynamic lighting conditions.

[0013] This invention provides a method for visually identifying cracks in bidirectional eccentrically compressed members due to abnormal stress, comprising:

[0014] Simultaneously acquire visible light images, long-wave infrared thermal images, and ambient light parameter data of the target area of ​​the bidirectional eccentric compression member. The ambient light parameter data includes light intensity data and color temperature data.

[0015] The visible light image and the long-wave infrared thermal imaging image are spatiotemporally aligned to generate pixel-level aligned multimodal image data pairs that are strictly corresponding in spatial coordinates and timestamps. The ambient light parameter data and the multimodal image data pairs are associated according to the timestamps to form a unified multi-source heterogeneous dataset.

[0016] This multi-source heterogeneous dataset provides a unified data input for the entire process of subsequent illumination compensation and cross-modal feature fusion. Each processing module completes the retrieval and collaborative analysis of corresponding modal data based on the multi-source heterogeneous dataset.

[0017] Based on ambient lighting parameters, the optimal enhancement operator and parameters are matched in the preset image enhancement operator mapping library to perform feedforward lighting compensation on the visible light image and generate an enhanced visible light image with normalized lighting.

[0018] The feedforward illumination compensation is as follows: using ambient illumination parameters as perturbation input, before the visible light image enters the subsequent crack segmentation process, illumination normalization is completed in one go by the matching enhancement operator and parameters.

[0019] The enhanced visible light image and the aligned long-wave infrared thermal image are used as dual-channel inputs and fed into a pre-trained dual-stream heterogeneous feature fusion convolutional neural network. Crack segmentation is achieved through cross-modal feature fusion, and a binarized crack mask image is output.

[0020] Morphological post-processing and topological analysis are performed on the binarized crack mask image to extract the geometric and topological parameters of each independent crack. The geometric and topological parameters include the crack length, maximum width, average width, total number of pixels, centroid coordinates, and orientation angle relative to the main axis of the component.

[0021] The extracted geometric and topological parameters of the crack are input into a pre-built knowledge base of crack-stress anomaly patterns in bidirectional eccentrically compressed components for matching and reasoning, to determine the type of stress anomaly that causes the current crack morphology, and the final identification result is output.

[0022] Preferably, the step of outputting a binary crack mask image representing the location and morphology of the crack includes:

[0023] The enhanced visible light image with normalized illumination and the long-wave infrared thermal imaging image are used as dual-channel inputs and fed into a pre-trained dual-stream heterogeneous feature fusion convolutional neural network model for crack segmentation and identification. The dual-stream heterogeneous feature fusion convolutional neural network model extracts high-frequency edge texture features from the enhanced visible light image and micro-temperature difference distribution features from the long-wave infrared thermal imaging image, and performs feature fusion in the deep layer of the network through a cross-modal attention mechanism.

[0024] Preferably, the simultaneous acquisition of visible light images, long-wave infrared thermal imaging images, and ambient light parameter data is achieved through an integrated multi-sensor front-end acquisition device;

[0025] The multi-sensor front-end acquisition device includes an industrial-grade global shutter visible light camera, an uncooled long-wave infrared focal plane array thermal imager, and an ambient light sensor composed of multiple photodiode arrays with different spectral response ranges. The three devices achieve microsecond-level synchronous acquisition through hardware trigger signals.

[0026] Preferably, the spatiotemporal alignment processing of the visible light image and the long-wave infrared thermal imaging image includes:

[0027] During the offline calibration phase, a specific checkerboard calibration plate that simultaneously covers the visible and infrared spectra is used to calculate and solidify the homography transformation matrix from the infrared image coordinate system to the visible image coordinate system.

[0028] During the real-time processing phase, the homography transformation matrix is ​​applied to perform perspective transformation on each frame of long-wave infrared thermal imaging image, so that its field of view and pixel coordinate system are completely aligned with the synchronously acquired visible light image.

[0029] Preferably, the image enhancement operator mapping library is a pre-calibrated query library pre-built to implement the feedforward illumination compensation, and its construction process includes:

[0030] In a controlled lighting laboratory, the light intensity and color temperature of the light source are systematically changed to simulate various typical lighting conditions at the construction site, covering the full range of environmental lighting parameters.

[0031] Under each fixed combination of illumination intensity and color temperature, visible light images of standard crack samples are acquired, and the images are processed using various image enhancement algorithms and different parameter combinations. The crack signal-to-noise ratio of each processing result is evaluated. The image enhancement algorithms include a contrast-limited adaptive histogram equalization algorithm, a multi-scale retinal enhancement algorithm, a wavelet transform-based image enhancement algorithm, and a generative adversarial network-based image deshading algorithm.

[0032] For each combination of light intensity and color temperature, the optimal enhancement algorithm that maximizes the signal-to-noise ratio of the crack and its corresponding optimal parameter combination are selected. A two-dimensional lookup table is constructed with light intensity and color temperature as a joint unique index and the optimal enhancement operator identifier and corresponding parameter set as storage content, forming the mapping library.

[0033] Preferably, the dual-stream heterogeneous feature fusion convolutional neural network model has a parallel dual-encoder-single-decoder structure;

[0034] One encoder branch employs a residual network structure to process the enhanced visible light image to extract deep crack edges and texture details; the other encoder branch employs a lightweight convolutional network structure to process the long-wave infrared thermal imaging image to extract subtle temperature difference features in the crack area caused by stress concentration or moisture infiltration.

[0035] At the end of the encoder path, the feature maps of the two branches are weighted and fused by the spatial and channel attention fusion module. The fused feature map is then fed into the decoder path, where upsampling and deconvolution operations are used to restore the image resolution step by step and generate pixel-level crack segmentation results.

[0036] Preferably, the training process of the dual-stream heterogeneous feature fusion convolutional neural network model includes:

[0037] Supervised training was conducted using a crack mask dataset with pixel-level precise annotations, which contains a large number of crack images of bidirectional eccentrically compressed components taken under different lighting and angles and their corresponding visible-infrared images.

[0038] The loss function of the dual-stream heterogeneous feature fusion convolutional neural network model is a linear combination of a weighted binary cross-entropy loss function and a Dessian similarity coefficient loss function, where the weight coefficients are used to balance the imbalance between positive and negative samples.

[0039] Preferably, the morphological post-processing and topological analysis of the binarized crack mask image include:

[0040] The opening operation is applied to remove isolated small-area pixels caused by noise, and the closing operation is applied to connect the broken crack segments.

[0041] Connectivity analysis is performed on the processed image to assign a unique identifier to each individual crack region;

[0042] The skeleton of each connected component is extracted, and the length of the crack is calculated based on the skeleton. The width of each point of the crack is calculated based on the distance transformation algorithm, and the maximum width and average width are finally determined. At the same time, the main direction angle of the crack is determined by Hough transform or principal component analysis.

[0043] Preferably, the knowledge base of crack-stress anomaly patterns of bidirectional eccentrically compressed components is constructed using a dual-path approach of machine learning unsupervised clustering mining combined with manual induction and verification based on structural mechanics theory. It uses finite element structural analysis simulation data as the basic dataset and physical loading experimental data as the verification and calibration dataset. The mapping rules stored inside map the combination of crack geometric and topological parameters with quantized judgment intervals and weight coefficients to the corresponding structural stress anomaly types.

[0044] The steps for constructing the knowledge base of crack-stress anomaly modes in bidirectional eccentrically compressed members include:

[0045] Data pairing and acquisition: During the simulation phase, different combinations of axial forces and bidirectional bending moments are applied to the standard bidirectional eccentric compression member model, and the structural stress data and corresponding crack geometric and topological parameters under each working condition are recorded simultaneously; during the physical experiment phase, the real concrete bidirectional eccentric compression member is subjected to graded loading, and image data of the crack development process and stress data under the corresponding load are collected simultaneously, and the geometric and topological parameters of the corresponding cracks are extracted; each set of synchronously collected stress data is bound with the geometric and topological parameters of the corresponding crack to form a standardized data pair of stress mode-crack feature;

[0046] Construction of standardized dataset: Outliers and duplicate samples are removed from the standardized data pairs to construct a crack feature dataset with stress labels;

[0047] Clustering and Initial Rule Generation: Based on the crack feature dataset with stress labels, the DBSCAN density clustering algorithm is used to perform unsupervised clustering of crack feature vectors to obtain multiple clusters corresponding to different combinations of crack geometric and topological parameters; each cluster is correlated with the bound stress labels, the dominant stress anomaly type corresponding to each cluster is statistically analyzed, and initial mapping rules are generated.

[0048] Rule verification and solidification: Combining the mechanical theory of bidirectional eccentric compression of concrete structures, the initial mapping rules are manually verified and corrected. The quantitative judgment interval and weight coefficient of the crack geometry and topology parameters corresponding to each mapping rule are clarified. Clustering results that do not conform to the mechanical principles are eliminated, and finally solidified mapping rules are formed.

[0049] In the mapping rules, the crack orientation angle is the core judgment parameter, accounting for 60% of the weight; the crack distribution pattern and spatial location are secondary judgment parameters, accounting for 30% of the weight; and the crack length and width are auxiliary judgment parameters, accounting for 10% of the weight. Specific mapping rules include:

[0050] Crack patterns with orientation angles between 40 and 50 degrees and a network-like cross-distribution are mapped as shear stress exceeding limits.

[0051] The crack pattern with a direction angle close to 0 degrees, parallel to the short side of the member and penetrating the height of the member section is mapped as a large eccentric compression bending moment overload.

[0052] The crack pattern that is concentrated at the corner of the component, radiates outwards in a radial pattern, and has its centroid located in the corner region is mapped as a local compressive stress concentration.

[0053] The crack pattern with a direction angle close to 90 degrees, parallel to the long side of the component, and distributed in the middle of the compression zone is mapped as small eccentric compression instability.

[0054] Preferably, the fusion formula of the cross-modal attention mechanism is:

[0055] ;

[0056] This is a three-dimensional feature map with dimensions H × W × number of channels C, output by the visible light branch. This is a three-dimensional feature map output from the infrared branch that has the same dimensions as the visible light branch feature map. The spatial-channel joint attention weight three-dimensional tensor is of the same dimension as the feature map. The weight value at each spatial location and in each channel dimension is dynamically calculated based on the local response intensity at the corresponding location of the two feature maps and the global statistical characteristics of the full map features.

[0057] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0058] 1. By actively sensing ambient lighting parameters and performing feedforward compensation, this invention fundamentally solves the problem of dynamic lighting interfering with crack image quality, improves the robustness and detection accuracy of the algorithm in harsh construction site environments such as low illumination, strong shadows, and mixed light sources, and reduces the false negative rate.

[0059] 2. This invention employs a dual-modal data fusion strategy of visible light and infrared thermal imaging, utilizing the complementary information of cracks under different physical modes. The sensitivity of long-wave infrared images to temperature differences enables them to detect early-stage cracks that are extremely fine on the surface or obscured by dust and difficult to identify under visible light, thereby improving the sensitivity and comprehensiveness of identification.

[0060] 3. The dual-stream heterogeneous feature fusion network constructed in this invention designs a dedicated feature extraction stream for the characteristics of different modal data, and achieves information fusion through a cross-modal attention mechanism. Compared with simple image stitching and input into a single-stream network, it can more deeply explore the correlation between modalities and improve the accuracy of crack segmentation.

[0061] 4. This invention goes beyond crack detection and measurement. By establishing a knowledge base mapping relationship between crack morphology and abnormal structural stress patterns, it achieves intelligent diagnosis from "what" to "why," providing on-site engineers with direct basis for structural safety status assessment and possessing higher engineering application value. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of the overall technical solution architecture of a visual identification method for abnormal stress in bidirectional eccentrically compressed components proposed in this invention.

[0063] Figure 2 This is a schematic diagram of the core principle framework of feedforward image enhancement and multimodal data alignment based on ambient lighting parameters in this invention;

[0064] Figure 3 This is a logical flowchart of the dual-stream heterogeneous feature fusion convolutional neural network model in this invention;

[0065] Figure 4 This is a flowchart illustrating the logical flow of crack geometry and topology parameter extraction and morphological post-processing in this invention.

[0066] Figure 5 This is a logical flowchart of the knowledge base reasoning and diagnosis of crack-stress anomaly mode in bidirectional eccentrically compressed components in this invention.

[0067] Figure 6 This is a schematic diagram of the multi-level interaction and data flow between the multi-sensor front-end acquisition device and the cloud processing unit in this invention. Detailed Implementation

[0068] Please refer to Figures 1 to 6 This invention provides a visual identification method for cracks in bidirectional eccentrically compressed components, aiming to solve the technical problems of blurred crack edges caused by dynamic lighting interference at construction sites, high missed detection rate under low illumination conditions, and inability to compensate for lighting in real time. This method integrates multispectral imaging, active ambient lighting perception and compensation mechanisms, and stress anomaly inference logic based on prior physical knowledge to construct a closed-loop identification process, achieving highly robust detection and causal diagnosis of cracks in components.

[0069] The method includes the following steps:

[0070] Simultaneously acquire visible light images, long-wave infrared thermal images, and ambient light parameter data of the target area of ​​the bidirectional eccentric compression member; the ambient light parameter data includes light intensity data and color temperature data.

[0071] The visible light image and the long-wave infrared thermal imaging image are spatiotemporally aligned to generate pixel-level aligned multimodal image data pairs that are strictly corresponding in spatial coordinates and timestamps. The ambient light parameter data and the multimodal image data pairs are associated according to the timestamps to form a unified multi-source heterogeneous dataset.

[0072] Based on the light intensity and color temperature values ​​in the ambient light parameter data, the image enhancement operator and its processing parameters corresponding to the current real-time lighting conditions are queried and matched in the preset component surface lighting model and image enhancement operator mapping library. The image enhancement operator is then used to perform targeted feedforward lighting compensation and feature enhancement processing on the visible light image in the multimodal image data pair to generate an enhanced visible light image with normalized lighting.

[0073] The feedforward illumination compensation is an open-loop compensation control method based on disturbances. Its core logic is as follows: using the real-time collected ambient light intensity and color temperature as the system disturbance input, before the visible light image enters the subsequent crack segmentation process, the optimal enhancement operator and parameters are directly matched through a pre-built mapping library to complete the illumination normalization processing of the visible light image in one go. Unlike the feedback illumination compensation method, which requires iterative adjustment of processing parameters based on the quality evaluation results of the enhanced image, the feedforward compensation in this step does not require closed-loop feedback iteration. Illumination normalization can be completed by a single table lookup and a single image processing, thus eliminating the impact of ambient light disturbances on image quality from the root.

[0074] The enhanced visible light image with normalized illumination and the long-wave infrared thermal imaging image are used as dual-channel inputs and fed into a pre-trained dual-stream heterogeneous feature fusion convolutional neural network model for crack segmentation and identification. The dual-stream heterogeneous feature fusion convolutional neural network model extracts high-frequency edge texture features from the enhanced visible light image and micro-temperature difference distribution features from the long-wave infrared thermal imaging image, respectively, and performs feature fusion through a cross-modal attention mechanism in the deep layer of the network, finally outputting a binary crack mask image representing the crack location and morphology.

[0075] Morphological post-processing and topological analysis are performed on the binarized crack mask image to extract the geometric and topological parameters of each independent crack. The geometric and topological parameters include the crack length, maximum width, average width, total number of pixels, centroid coordinates, and orientation angle relative to the main axis of the component.

[0076] The extracted geometric and topological parameters of the crack are input into a pre-built knowledge base of crack-stress anomaly patterns in bidirectional eccentrically compressed components for matching and reasoning, to determine the type of stress anomaly that causes the current crack morphology, and the final identification result is output.

[0077] In the aforementioned steps, the simultaneous acquisition of visible light images, long-wave infrared thermal images, and ambient light parameter data is achieved through an integrated multi-sensor front-end acquisition device. This device includes an industrial-grade global shutter visible light camera, an uncooled long-wave infrared focal plane array thermal imager, and an ambient light sensor composed of multiple photodiode arrays with different spectral response ranges.

[0078] The three devices achieve microsecond-level synchronized acquisition via hardware trigger signals, ensuring strict alignment of all data in the time dimension. A visible light camera is used to capture macroscopic texture and color information of the component surface, with a resolution greater than 4 megapixels and a frame rate greater than 30 Hz.

[0079] Long-wave infrared thermal imagers operate in the 8 to 14 micrometer band, with a spatial resolution of 160 x 120 pixels or more and a temperature sensitivity better than 0.05 degrees Celsius. They are used to detect minute temperature differences on the surface of components caused by stress concentration or moisture penetration.

[0080] The ambient light sensor consists of at least three silicon-based photodiodes with different spectral response curves, covering the visible light blue, green and red bands respectively. Through weighted calculation, it can accurately invert the current ambient light intensity and correlated color temperature value. Its sampling frequency is greater than 100 Hz to adapt to the rapidly changing lighting conditions at the construction site.

[0081] After data acquisition is completed, spatiotemporal alignment processing is performed. This process consists of two stages: offline calibration and online alignment.

[0082] During the offline calibration phase, a specific checkerboard calibration plate covering both visible and infrared spectra was used to acquire multiple sets of synchronized image pairs in a laboratory environment. The extrinsic parameter matrix between the visible light camera and the infrared thermal imager was calculated using the Zhang Zhengyou calibration method, thereby deriving the homography transformation matrix from the infrared image coordinate system to the visible light image coordinate system. This matrix was permanently stored in the system memory as a reference for subsequent real-time processing.

[0083] During the online alignment phase, the system applies the aforementioned homography transformation matrix to each frame of the acquired long-wave infrared thermal imaging image and performs perspective transformation to ensure that its field of view, viewing angle center, and pixel coordinate system completely coincide with the synchronously acquired visible light image.

[0084] The transformed infrared image and the original visible light image are spatially aligned at the pixel level, and their timestamps are kept consistent through a hardware synchronization mechanism, thus forming a strictly spatiotemporally aligned multimodal image data pair.

[0085] Subsequently, the system binds the light intensity and color temperature data output by the ambient light sensor to this multimodal image data pair based on the same timestamp, forming a unified data unit containing three heterogeneous data types, providing complete input for subsequent processing.

[0086] The system retrieves visible light image data and illumination parameter data from a multi-source heterogeneous dataset, completes operator matching and image enhancement processing for feedforward illumination compensation, and temporarily stores infrared thermal imaging image data in the dataset for subsequent cross-modal fusion.

[0087] This process relies on a pre-built library of image enhancement operator maps.

[0088] The construction process of this mapping library is as follows:

[0089] This mapping library is a pre-calibrated query library built in advance to implement the aforementioned feedforward illumination compensation. Its construction process is as follows:

[0090] Inside the controlled lighting laboratory, an adjustable light source system is set up to systematically adjust the light intensity from 10 lux to 100,000 lux and the color temperature from 2,000 Kelvin to 7,000 Kelvin, fully covering the full range of ambient lighting parameters collected on site, simulating various typical lighting scenarios at the construction site, from weak light at night, diffuse light on cloudy days, strong direct light at noon to mixed artificial light sources.

[0091] Under each fixed combination of illumination intensity and color temperature, a set of standard crack samples (including prefabricated cracks of different widths, orientations, and depths) were imaged in visible light. Various image enhancement algorithms and different parameter combinations were then applied to process the acquired images. The image enhancement algorithms included a contrast-limited adaptive histogram equalization algorithm, a multi-scale retinal enhancement algorithm, a wavelet transform-based image enhancement algorithm, and a generative adversarial network-based image deshading algorithm.

[0092] Each algorithm was run under different parameter combinations, and the crack signal-to-noise ratio of its output image was quantitatively scored by a combination of manual annotation and automatic evaluation to clarify the enhancement effect of different algorithms and parameters under corresponding lighting conditions.

[0093] For each specific combination of light intensity and color temperature, the optimal enhancement algorithm that maximizes the signal-to-noise ratio of the crack and its corresponding optimal parameter combination are selected, and a two-dimensional lookup table is constructed. The lookup table uses light intensity and color temperature as joint unique index keys and the optimal enhancement operator identifier and corresponding parameter set as storage values, ultimately forming the mapping library.

[0094] The mapping library pre-calibrates the optimal enhancement parameters under full-range illumination conditions, enabling the optimal processing parameters to be directly matched based on the collected ambient illumination parameters during real-time on-site processing without the need for parameter iteration optimization. This provides data support for single-pass and open-loop processing of feedforward illumination compensation, ensuring the real-time performance and stability of the compensation process.

[0095] In actual operation, after the system reads the ambient light parameters synchronously acquired in the current frame, it directly performs a table lookup operation in the mapping library to match the optimal image enhancement operator and parameters that uniquely correspond to the current lighting conditions. The system then immediately applies the operator and parameters to the visible light image of the current frame without adjusting any parameters based on the image processing results, generating an enhanced visible light image with normalized illumination and completing the entire feedforward illumination compensation process.

[0096] The system retrieves spatiotemporally aligned long-wave infrared thermal imaging images from a multi-source heterogeneous dataset, and combines them with enhanced visible light images that have undergone illumination compensation to form standardized dual-channel input data, which is then fed into a dual-stream heterogeneous feature fusion convolutional neural network model.

[0097] This image has eliminated the interference of the original lighting in terms of brightness, contrast and color consistency, and the clarity of the crack edges has been improved.

[0098] Subsequently, the system uses the illumination-normalized enhanced visible light image and the spatiotemporally aligned long-wave infrared thermal image as dual-channel inputs, feeding them into a pre-trained dual-stream heterogeneous feature fusion convolutional neural network model. This dual-stream heterogeneous feature fusion convolutional neural network model employs a parallel dual-encoder-single-decoder architecture.

[0099] The dual-stream heterogeneous feature fusion convolutional neural network model sets a uniform input image size of 512×512 pixels. Before input, the dual-channel images undergo standardization preprocessing: for enhanced visible light images, pixel values ​​are normalized to the [0,1] interval, and then channel standardization is performed using the mean and standard deviation of the ImageNet dataset; for long-wave infrared thermal imaging images, temperature values ​​are first linearly mapped to the [0,255] grayscale interval, and then normalized to the [0,1] interval to complete the standardization process; for input images with inconsistent sizes, bilinear interpolation is used to scale them to 512×512 pixels to ensure that the input data meets the model's size requirements.

[0100] The visible light branch encoder adopts a deep residual network structure, which contains five residual blocks. Each block consists of two convolutional layers and a batch normalization layer, which are used to extract the high-frequency edges, texture details and local geometric features of the crack step by step.

[0101] The infrared branch encoder adopts a lightweight convolutional network structure, which contains four convolutional layers and a max pooling layer stacked alternately. It is specifically designed to extract the subtle temperature difference distribution characteristics in the crack area caused by internal stress concentration or moisture evaporation. Its design balances computational efficiency and thermal sensitivity.

[0102] The two encoding paths converge at the deep feature map level and enter the spatial and channel attention fusion module.

[0103] This module first calculates the spatial attention weight and channel attention weight for the two feature maps respectively, and then performs feature fusion by weighted summation, as expressed by the formula:

[0104] ;

[0105] The feature map output by the visible light branch. The feature map output by the infrared branch. The spatial-channel joint attention weights are a three-dimensional tensor with the same dimension as the feature map. The tensor dimension is the same as the feature map. Figure 1 One-to-one correspondence, covering the full spatial dimension of height H and width W and the total number of channels C. The weight value of each spatial location and each channel dimension is calculated based on the local response intensity of the corresponding location of the two feature maps. The channel dimension weight is calculated by combining the global statistical characteristics of the full image features of the two feature maps. The spatial dimension weight and the channel dimension weight are then dynamically obtained by fusing them together, realizing differentiated weighted fusion for different spatial locations and different feature channels.

[0106] The fused feature map is fed into the decoder path, which consists of four upsampling layers and deconvolution layers, progressively restoring the image resolution and introducing skip connections at each level to preserve shallow detail information.

[0107] Finally, the decoder outputs a single-channel probability map of the same size as the input image. After thresholding, a binary crack mask image is generated, in which the area with a pixel value of 1 represents the presence of cracks, and the area with a pixel value of zero represents the background.

[0108] The pre-training process of the dual-stream heterogeneous feature fusion convolutional neural network model was completed using a multimodal dataset of cracks in bidirectional eccentrically compressed components containing 12,000 sets of valid labeled samples. This dataset covers visible light-infrared image pairs of bidirectional eccentrically compressed components under different illumination intensities, shooting angles, and stress conditions. Among them, 85% of the samples are crack images of real engineering components collected on-site, and 15% of the samples are crack images collected from graded loading tests of precast components in the laboratory. All samples were pixel-level crack labeled using the LabelMe tool to generate a binarized crack mask gold standard image corresponding to the input image size.

[0109] The model training employs a five-fold cross-validation method, randomly dividing the dataset into five non-overlapping subsets. Four subsets are selected as the training set and one subset as the validation set in each iteration, repeating this process five times to ensure the model's generalization ability. The training process is implemented using the PyTorch deep learning framework, with an NVIDIA RTX 3090 GPU and 16GB of VRAM. The AdamW optimizer is used, with an initial learning rate of 1e-4, a weight decay coefficient of 1e-5, a batch size of 16, and a maximum training iteration count of 100 epochs. An early stopping mechanism is employed during training: when the Dessian similarity coefficient of the validation set does not improve for 10 consecutive epochs, training is automatically terminated, and the current optimal model weights are saved to prevent overfitting.

[0110] Loss function for model training A linear combination of the weighted binary cross-entropy loss function and the Dessie similarity coefficient loss function is used, and the specific expression is as follows: ,in This is the weighting coefficient, set to 0.4, used to balance the imbalance between positive and negative samples; For weighted binary cross-entropy loss, the weight of positive samples is set to 10 and the weight of negative samples is set to 1, in order to adapt to the sample characteristics where the proportion of crack pixels is extremely low. The loss factor is the Dessian similarity coefficient, used to improve the model's segmentation and overlap of crack regions.

[0111] After the model is trained, a separate test set is used for final accuracy verification. The test set contains 2,400 samples that were not used in training and verification. The verification metrics include pixel accuracy, recall, precision and Dessian similarity coefficient.

[0112] After obtaining the binary crack mask image, the system performs morphological post-processing and topological analysis.

[0113] First, an opening operation is performed using a circular kernel with a 3x3 structuring element to remove isolated noise points with an area of ​​less than 9 pixels.

[0114] The same structuring element is then used to perform a closing operation to connect broken crack segments caused by image noise or segmentation errors. The processed image undergoes 8-connected component analysis, assigning a unique integer identifier to each individual crack region.

[0115] For each connected component, the system executes a skeleton extraction algorithm to obtain the centerline of the crack. Based on this centerline, a distance transformation algorithm is used to calculate the shortest distance from each point on the centerline to the original crack boundary. Twice this distance is the crack width at that point.

[0116] This allows us to calculate the maximum width, average width, and total number of pixels for each crack. The crack length is calculated using the cumulative Euclidean distance between the skeleton pixels.

[0117] The centroid coordinates are obtained by the arithmetic mean of the coordinates of all pixels in the connected domain.

[0118] The direction angle of the crack is determined by principal component analysis: the coordinates of all pixels of the crack are used to form a two-dimensional dataset, its covariance matrix is ​​calculated, the direction of the first principal component is taken as the main direction of the crack, and the angle between this direction and the preset principal axis of the component is the direction angle.

[0119] All geometric and topological parameters are stored in a structured manner, forming crack feature vectors.

[0120] Finally, the system inputs the crack feature vector into a pre-built knowledge base of crack-stress anomaly modes in bidirectional eccentrically compressed components for matching and inference. This knowledge base is built based on a large amount of finite element structural simulation and physical loading experimental data.

[0121] The knowledge base is constructed using a dual approach: unsupervised clustering mining via machine learning and manual induction and verification of structural mechanics theory. It uses full-volume finite element simulation data as the basic dataset and measured data from physical experiments as the verification and calibration dataset to extract and solidify the mapping rules between crack parameters and stress modes.

[0122] During the simulation phase, different combinations of axial forces and bidirectional bending moments were applied to a standard bidirectional eccentrically compressed member model, and the resulting crack morphology parameters were recorded.

[0123] During the experimental phase, real concrete components were subjected to graded loading, and crack development process and load data were collected simultaneously.

[0124] After completing the simulation and experimental data acquisition, the data is first standardized and preprocessed: the axial force, bidirectional bending moment, shear stress, compressive stress and other stress data collected synchronously in the simulation and experiment are bound with the geometric topological parameters such as crack length, width, orientation angle, distribution pattern and centroid position extracted under the corresponding working conditions to form stress mode-crack feature standardized data pairs; after removing outliers and duplicate samples in the data pairs, a crack feature dataset with stress labels is constructed.

[0125] Based on the aforementioned dataset, the DBSCAN density clustering algorithm was used to perform unsupervised clustering of crack feature vectors, resulting in different clusters. Each cluster corresponds to a set of highly similar crack geometric topology parameters. The clusters were then correlated with their associated stress labels, and the dominant stress anomaly types corresponding to each cluster were statistically analyzed to form initial mapping rules. In conjunction with the mechanical theory of bidirectional eccentric compression of concrete structures, the initial mapping rules were manually verified and corrected to clarify the crack parameter threshold ranges corresponding to each rule. Clustering results that did not conform to the mechanical principles were eliminated, and finally, solidified mapping rules were formed.

[0126] Through clustering and rule induction, the following mapping rules are established:

[0127] When the crack direction angle is between 40 and 50 degrees and it is distributed in a network pattern, it is determined that the shear stress exceeds the limit; when the crack direction angle is close to zero degrees (parallel to the short side of the member) and it runs through the entire section height, it is determined that the large eccentric compression bending moment is overloaded.

[0128] When cracks are concentrated at the corners of a component and radiate outwards, with the centroid located in the corner region, it is determined to be a localized compressive stress concentration; when the crack direction angle is close to 90 degrees (parallel to the long side of the component) and distributed in the middle of the compression zone, it is determined to be a small eccentric compression instability.

[0129] Each mapping rule has a corresponding quantitative judgment range and weight coefficient for crack geometric topology parameters. The crack orientation angle is the core judgment parameter, accounting for 60% of the weight; the crack distribution pattern and spatial location are secondary judgment parameters, accounting for 30% of the weight; and the crack length and width are auxiliary judgment parameters, accounting for 10% of the weight. This achieves accurate quantitative correspondence between crack geometric topology parameters and stress anomaly modes.

[0130] The system will logically match the crack parameters extracted in real time with the above rules one by one, calculate the matching degree score of each rule, select the stress anomaly type corresponding to the rule with the highest score and greater than the preset threshold as the final diagnosis result, and output it to the user interface or structural health monitoring platform.

[0131] Throughout the entire process, the data flow strictly follows the chronological order and logical dependencies.

[0132] The multi-sensor front-end acquisition device continuously outputs synchronous data streams, the spatiotemporal alignment module generates aligned image pairs in real time, the illumination compensation module dynamically calls enhancement strategies based on environmental parameters, the dual-stream network model performs high-precision segmentation, the post-processing module extracts quantized features, and the knowledge base completes the final diagnosis.

[0133] The modules communicate efficiently through memory sharing or message queues, ensuring that the end-to-end latency is controlled within 500 milliseconds, which meets the real-time monitoring needs of the construction site.

[0134] The system has an exception handling mechanism: if the ambient light sensor fails, the default neutral light enhancement strategy will be activated.

[0135] If the quality of the infrared image is less than the threshold, single-modal identification is performed solely by enhancing the visible light image, and the diagnostic confidence is reduced. If the crack parameters cannot match any knowledge base rules, it is marked as an "unknown abnormal mode" and a manual review process is triggered.

[0136] The method described in this embodiment fully realizes the automated processing of the entire chain from raw data acquisition to structural stress anomaly diagnosis, overcomes dynamic lighting interference, and improves the accuracy of crack identification and engineering practicality.

[0137] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish an entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0138] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for visually identifying cracks in bidirectional eccentrically compressed members due to abnormal stress, characterized in that, include: Simultaneously acquire visible light images, long-wave infrared thermal images, and ambient light parameter data of the target area of ​​the bidirectional eccentric compression member. The ambient light parameter data includes light intensity data and color temperature data. The visible light image and the long-wave infrared thermal imaging image are spatiotemporally aligned to generate pixel-level aligned multimodal image data pairs that are strictly corresponding in spatial coordinates and timestamps. The ambient light parameter data and the multimodal image data pairs are associated according to the timestamps to form a unified multi-source heterogeneous dataset. This multi-source heterogeneous dataset provides a unified data input for the entire process of subsequent illumination compensation and cross-modal feature fusion. Each processing module completes the retrieval and collaborative analysis of corresponding modal data based on the multi-source heterogeneous dataset. Based on ambient lighting parameters, the optimal enhancement operator and parameters are matched in the preset image enhancement operator mapping library to perform feedforward lighting compensation on the visible light image and generate an enhanced visible light image with normalized lighting. The feedforward illumination compensation is as follows: using ambient illumination parameters as perturbation input, before the visible light image enters the subsequent crack segmentation process, illumination normalization is completed in one go by the matching enhancement operator and parameters. The enhanced visible light image and the aligned long-wave infrared thermal image are used as dual-channel inputs and fed into a pre-trained dual-stream heterogeneous feature fusion convolutional neural network. Crack segmentation is achieved through cross-modal feature fusion, and a binarized crack mask image is output. Morphological post-processing and topological analysis are performed on the binarized crack mask image to extract the geometric and topological parameters of each independent crack. The geometric and topological parameters include the crack length, maximum width, average width, total number of pixels, centroid coordinates, and orientation angle relative to the main axis of the component. The extracted geometric and topological parameters of the crack are input into a pre-built knowledge base of crack-stress anomaly patterns in bidirectional eccentrically compressed components for matching and reasoning, to determine the type of stress anomaly that causes the current crack morphology, and the final identification result is output. The knowledge base of crack-stress anomaly patterns of bidirectional eccentrically compressed components is constructed using a dual-path approach: unsupervised clustering mining through machine learning combined with manual induction and verification based on structural mechanics theory. It uses finite element structural analysis simulation data as the basic dataset and physical loading experimental data as the verification and calibration dataset. The mapping rules stored inside map the combination of crack geometric and topological parameters with quantized judgment intervals and weight coefficients to the corresponding structural stress anomaly types. The steps for constructing the knowledge base of crack-stress anomaly modes in bidirectional eccentrically compressed members include: Data pairing and acquisition: During the simulation phase, different combinations of axial forces and bidirectional bending moments are applied to the standard bidirectional eccentric compression member model, and the structural stress data and corresponding crack geometric and topological parameters under each working condition are recorded simultaneously; during the physical experiment phase, the real concrete bidirectional eccentric compression member is subjected to graded loading, and image data of the crack development process and stress data under the corresponding load are collected simultaneously, and the geometric and topological parameters of the corresponding cracks are extracted; each set of synchronously collected stress data is bound with the geometric and topological parameters of the corresponding crack to form a standardized data pair of stress mode-crack feature; Construction of standardized dataset: Outliers and duplicate samples are removed from the standardized data pairs to construct a crack feature dataset with stress labels; Clustering and Initial Rule Generation: Based on the crack feature dataset with stress labels, the DBSCAN density clustering algorithm is used to perform unsupervised clustering of crack feature vectors to obtain multiple clusters corresponding to different combinations of crack geometric and topological parameters; each cluster is correlated with the bound stress labels, the dominant stress anomaly type corresponding to each cluster is statistically analyzed, and initial mapping rules are generated. Rule verification and solidification: Combining the mechanical theory of bidirectional eccentric compression of concrete structures, the initial mapping rules are manually verified and corrected. The quantitative judgment interval and weight coefficient of the crack geometry and topology parameters corresponding to each mapping rule are clarified. Clustering results that do not conform to the mechanical principles are eliminated, and finally solidified mapping rules are formed. In the mapping rules, the crack orientation angle is the core judgment parameter, accounting for 60% of the weight; the crack distribution pattern and spatial location are secondary judgment parameters, accounting for 30% of the weight; and the crack length and width are auxiliary judgment parameters, accounting for 10% of the weight. Specific mapping rules include: Crack patterns with orientation angles between 40 and 50 degrees and a network-like cross-distribution are mapped as shear stress exceeding limits. The crack pattern with a direction angle close to 0 degrees, parallel to the short side of the member and penetrating the height of the member section is mapped as a large eccentric compression bending moment overload. The crack pattern that is concentrated at the corner of the component, radiates outwards in a radial pattern, and has its centroid located in the corner region is mapped as a local compressive stress concentration. The crack pattern with a direction angle close to 90 degrees, parallel to the long side of the component, and distributed in the middle of the compression zone is mapped as small eccentric compression instability.

2. The method for visually identifying cracks in bidirectional eccentrically compressed members according to claim 1, characterized in that, The steps to output a binary crack mask image representing the location and morphology of the crack include: The enhanced visible light image with normalized illumination and the long-wave infrared thermal imaging image are used as dual-channel inputs and fed into a pre-trained dual-stream heterogeneous feature fusion convolutional neural network model for crack segmentation and identification. The dual-stream heterogeneous feature fusion convolutional neural network model extracts high-frequency edge texture features from the enhanced visible light image and micro-temperature difference distribution features from the long-wave infrared thermal imaging image, and performs feature fusion in the deep layer of the network through a cross-modal attention mechanism.

3. The method for visually identifying cracks in bidirectional eccentrically compressed members according to claim 1, characterized in that, Simultaneous acquisition of visible light images, long-wave infrared thermal imaging images, and ambient light parameter data is achieved through an integrated multi-sensor front-end acquisition device; The multi-sensor front-end acquisition device includes an industrial-grade global shutter visible light camera, an uncooled long-wave infrared focal plane array thermal imager, and an ambient light sensor composed of multiple photodiode arrays with different spectral response ranges. The three devices achieve microsecond-level synchronous acquisition through hardware trigger signals.

4. The method for visually identifying cracks in bidirectional eccentrically compressed members according to claim 1, characterized in that, Spatiotemporal alignment processing of the visible light image and the long-wave infrared thermal imaging image includes: During the offline calibration phase, a specific checkerboard calibration plate that simultaneously covers the visible and infrared spectra is used to calculate and solidify the homography transformation matrix from the infrared image coordinate system to the visible image coordinate system. During the real-time processing phase, the homography transformation matrix is ​​applied to perform perspective transformation on each frame of long-wave infrared thermal imaging image, so that its field of view and pixel coordinate system are completely aligned with the synchronously acquired visible light image.

5. The method for visually identifying cracks in bidirectional eccentrically compressed members according to claim 1, characterized in that, The image enhancement operator mapping library is a pre-calibrated query library pre-built to implement the feedforward illumination compensation, and its construction process includes: In a controlled lighting laboratory, the light intensity and color temperature of the light source are systematically changed to simulate various typical lighting conditions at the construction site, covering the full range of environmental lighting parameters. Under each fixed combination of illumination intensity and color temperature, visible light images of standard crack samples are acquired, and the images are processed using various image enhancement algorithms and different parameter combinations. The crack signal-to-noise ratio of each processing result is evaluated. The image enhancement algorithms include a contrast-limited adaptive histogram equalization algorithm, a multi-scale retinal enhancement algorithm, a wavelet transform-based image enhancement algorithm, and a generative adversarial network-based image deshading algorithm. For each combination of light intensity and color temperature, the optimal enhancement algorithm that maximizes the signal-to-noise ratio of the crack and its corresponding optimal parameter combination are selected. A two-dimensional lookup table is constructed with light intensity and color temperature as a joint unique index and the optimal enhancement operator identifier and corresponding parameter set as storage content, forming the mapping library.

6. The method for visually identifying cracks in bidirectional eccentrically compressed members according to claim 2, characterized in that, The dual-stream heterogeneous feature fusion convolutional neural network model has a parallel dual encoder-single decoder structure; One encoder branch employs a residual network structure to process the enhanced visible light image to extract deep crack edges and texture details; the other encoder branch employs a lightweight convolutional network structure to process the long-wave infrared thermal imaging image to extract subtle temperature difference features in the crack area caused by stress concentration or moisture infiltration. At the end of the encoder path, the feature maps of the two branches are weighted and fused by the spatial and channel attention fusion module. The fused feature map is then fed into the decoder path, where upsampling and deconvolution operations are used to restore the image resolution step by step and generate pixel-level crack segmentation results.

7. The method for visually identifying cracks in bidirectional eccentrically compressed members according to claim 6, characterized in that, The training process of the dual-stream heterogeneous feature fusion convolutional neural network model includes: Supervised training was conducted using a crack mask dataset with pixel-level precise annotations, which contains a large number of crack images of bidirectional eccentrically compressed components taken under different lighting and angles and their corresponding visible-infrared images. The loss function of the dual-stream heterogeneous feature fusion convolutional neural network model is a linear combination of a weighted binary cross-entropy loss function and a Dessian similarity coefficient loss function, where the weight coefficients are used to balance the imbalance between positive and negative samples.

8. The method for visually identifying cracks in bidirectional eccentrically compressed members according to claim 2, characterized in that, Morphological post-processing and topological analysis are performed on the binarized crack mask image, including: The opening operation is applied to remove isolated small-area pixels caused by noise, and the closing operation is applied to connect the broken crack segments. Connectivity analysis is performed on the processed image to assign a unique identifier to each individual crack region; The skeleton of each connected component is extracted, and the length of the crack is calculated based on the skeleton. The width of each point of the crack is calculated based on the distance transformation algorithm, and the maximum width and average width are finally determined. At the same time, the main direction angle of the crack is determined by Hough transform or principal component analysis.

9. The method for visually identifying cracks in bidirectional eccentrically compressed members according to claim 2, characterized in that, The fusion formula for the cross-modal attention mechanism is as follows: ； This is a three-dimensional feature map with dimensions H × W × number of channels C, output by the visible light branch. This is a three-dimensional feature map output from the infrared branch that has the same dimensions as the visible light branch feature map. The spatial-channel joint attention weight three-dimensional tensor is of the same dimension as the feature map. The weight value at each spatial location and in each channel dimension is dynamically calculated based on the local response intensity at the corresponding location of the two feature maps and the global statistical characteristics of the full map features.

Images