An intelligent detection system for defects of steel structure based on image recognition

Abstract

This invention discloses an intelligent steel structure defect detection system based on image recognition, belonging to the field of visual image recognition technology. The system includes: an image signal acquisition module that simultaneously acquires high-definition image sequences of the target area surface and microseismic signals; a safety benchmark comparison module that generates a visual feature set and compares it with a preset safety benchmark; a microseismic activity threshold comparison module that extracts a microseismic feature set and compares it with a microseismic activity threshold; a feature deviation calculation module that constructs a coupled feature group and calculates the deviation degree; a latent fatigue defect judgment module that judges defects according to conditions; and an early warning information output module that assesses lifespan reduction and outputs an early warning. This invention solves the technical problem that traditional visual inspection methods struggle to detect early fatigue damage that has already appeared internally but appears on the intact surface of a steel structure. It achieves the technical effect of accurately identifying internal fatigue damage invisible to the naked eye, while simultaneously distinguishing damage types, calculating the remaining lifespan of components, and updating the structural digital model, thus reducing the probability of missed or false detections.

Description

Technical Field

[0001] This invention relates to the field of visual image recognition technology, and more specifically to an intelligent detection system for steel structure defects based on image recognition. Background Technology

[0002] Steel structures are widely used in bridges, factories, wind turbine towers, and other projects. Hidden fatigue damage within components can easily lead to structural failure. Timely and accurate identification of potential hazards is crucial for the safe operation and maintenance of structures. Visual inspection is the mainstream method for screening external defects in steel structures. Existing technologies mostly employ single-vision image recognition for inspection, relying solely on image texture to determine surface damage, or using micro-vibration sensors alone to collect internal signals. While these methods are effective in detecting visible defects such as surface cracks and corrosion, in-service steel structures often exhibit hidden fatigue damage—appearing intact on the outside but with continuously expanding internal micro-cracks. Single-vision inspection cannot capture information about internal stress release in the material. The lack of a unified spatiotemporal reference and coupled correlation logic between the two types of data makes it difficult to distinguish between hidden defects that appear normal on the surface but are internally deteriorated. The inspection data is incomplete, with a high rate of false positives and false negatives, failing to achieve accurate identification of hidden fatigue defects in steel structures, life assessment, and risk warning. Summary of the Invention

[0003] This application provides an intelligent steel structure defect detection system based on image recognition, which solves the technical problem that traditional visual inspection methods are unable to detect early fatigue damage that has already occurred inside steel structures even when the surface is intact.

[0004] In view of the above problems, this application provides an intelligent detection system for steel structure defects based on image recognition.

[0005] This application provides an intelligent detection system for steel structure defects based on image recognition, the system comprising: The image signal acquisition module is used to simultaneously acquire high-resolution image sequences of the target area and microseismic signals rigidly connected to the target area during the application of service loads to the steel structure components. The safety benchmark comparison module is used to perform texture structure analysis on the high-resolution image sequences, generate a set of visual features characterizing the surface integrity, and compare the set of visual features with a preset safety benchmark. The microseismic activity threshold comparison module is used to perform time-frequency transformation processing on the microseismic signals, extract a set of microseismic features characterizing the stress release properties within the material, and compare the set of microseismic features with a preset microseismic activity threshold. The system includes a comparison module; a feature deviation calculation module, used to map the visual feature set and the microseismic feature set to the same multidimensional space, construct a coupled feature group, and calculate the deviation degree of the two types of features in the coupled feature group; a latent fatigue defect determination module, used to determine that there is a latent fatigue defect in the target area when the visual feature set is within a preset safety benchmark range, while the microseismic feature set exceeds a preset microseismic activity threshold, and the deviation degree is greater than a preset risk threshold; and a warning information output module, used to assess the remaining life reduction of the target area based on the energy accumulation trend in the microseismic feature set, and output warning information.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages: This application simultaneously acquires surface images and microseismic signals of steel structures under service load conditions, analyzes and extracts surface texture features and internal vibration features, maps these two types of features to the same multi-dimensional space to calculate the degree of deviation from the state, and identifies internal hidden damage by combining multiple judgment conditions. Based on the vibration signal characteristics, it distinguishes damage types and calculates the remaining life of the component, simultaneously updating the stress parameters of the steel structure digital model. This allows for the accurate identification of internal fatigue hazards that appear intact, making the detection of hidden defects and life assessment results of steel structures more accurate and reliable. It achieves the technical effect of accurately identifying internal fatigue damage invisible to the naked eye, distinguishing damage types, calculating the remaining life of the component, and updating the structural digital model, thus reducing the probability of missed or false detections. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 This is a schematic diagram of a steel structure defect intelligent detection system based on image recognition, provided in an embodiment of this application.

[0009] Figure 2This is a flowchart illustrating the process of calculating the degree of deviation between two types of features in a coupled feature group in an intelligent steel structure defect detection system based on image recognition, as provided in an embodiment of this application.

[0010] Figure labeling: Image signal acquisition module 10, safety benchmark comparison module 20, microseismic activity threshold comparison module 30, feature deviation degree calculation module 40, latent fatigue defect judgment module 50, early warning information output module 60. Detailed Implementation

[0011] This application provides an intelligent steel structure defect detection system based on image recognition, which solves the technical problem that traditional visual inspection methods are unable to detect early fatigue damage that has already occurred inside steel structures even when the surface is intact.

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0013] Examples, such as Figure 1 As shown, this application provides an intelligent detection system for steel structure defects based on image recognition, the system comprising: The image signal acquisition module 10 is used to simultaneously acquire high-definition image sequences of the target area and microseismic signals that are rigidly connected to the target area during the application of service loads to the steel structure components.

[0014] In this embodiment, the microseismic signal is a weak, high-frequency vibration signal released outward in the form of transient elastic stress waves when the internal micro-damage of the steel structure under service loads, such as dislocation movement or the initiation and propagation of microcracks.

[0015] Specifically, during the data acquisition phase, an industrial area scan camera is selected as the image acquisition device, and a piezoelectric acoustic emission sensor is selected as the micro-vibration signal acquisition device. For example, a mainstream industrial-grade CMOS area scan camera can be used, with the specific model selected adaptably according to the detection accuracy requirements and ambient lighting conditions. The industrial area scan camera is mounted directly opposite the target area of ​​the steel structure, with the camera lens facing the surface of the component to completely cover the area to be inspected. The vertical distance between the lens and the surface of the steel structure component is set to 1.5 meters, which can be adaptively adjusted within the range of 1.0 meter to 2.5 meters depending on the actual size of the area to be inspected and the required spatial resolution. The camera mounting bracket is fixed to a rigid base on the site using expansion bolts, with a tightening torque of no less than 30 N·m to ensure the stability of the bracket and prevent equipment vibration during the shooting process.

[0016] The piezoelectric acoustic emission sensor uses a wide-frequency response piezoelectric ceramic sensor, and its resonant frequency range can be selected according to the material properties of the steel structure being measured and the expected defect type. For example, Vaseline is used as the acoustic wave conduction coupling agent for the piezoelectric acoustic emission sensor. After the coupling agent is evenly applied to the sensor's detection end face, it is pressed tightly against the outer wall of the target area of ​​the steel structure. A stainless steel clamp is then used to secure the sensor along its outer perimeter, achieving rigid bonding. The clamp tightening torque is set to 15 N·m, which can be slightly adjusted according to the sensor size to ensure no relative slippage between the sensor's detection end face and the outer wall of the component, achieving a rigid connection in an acoustic sense and ensuring high-fidelity transmission of micro-vibration signals. The sensor placement point and the camera's shooting center are located on the same component cross-section, and the distance between them along the component's length is controlled within 0.05 meters, ensuring complete spatial correspondence between the image acquisition area and the vibration acquisition area. The spatial spacing between the sensor and the camera can be adaptively adjusted according to the component's cross-sectional dimensions, but it must be ensured that the sensing areas of the two types of sensors spatially overlap.

[0017] The service load was applied using an electro-hydraulic servo fatigue testing machine. A sine wave was used for the loading waveform, with the example loading frequency set to 5 Hz and the stress ratio set to 0.1. The maximum load was calculated based on the yield strength and cross-sectional area of ​​the tested steel structure to ensure the component could withstand alternating fatigue loads within its elastic range. The data acquisition process continuously followed the application of the service load on the steel structure. Surface morphology changes under load were synchronously recorded within image frames, and vibration waveforms generated by stress release within the component were synchronously captured and output by piezoelectric sensors. For in-service steel structures where a testing machine could not be used, the service load could be directly taken from the natural alternating loads experienced by the component under actual working conditions. The data acquisition was completed during the duration of the actual service conditions, similarly satisfying the condition of synchronous acquisition during the service load application process.

[0018] After the equipment is powered on, a synchronous trigger circuit is established. The industrial area scan camera and the piezoelectric acoustic emission sensor share the same synchronous signal generator to output trigger pulses. The synchronous signal generator has a built-in high-precision temperature-controlled crystal oscillator as a globally unified timing clock source, outputting a standard timestamp for subsequent timing calibration. When the synchronous signal generator outputs a single pulse signal, it synchronously drives both types of equipment to start acquisition, setting unified acquisition parameters. The industrial area scan camera is set to a sampling frame rate of 30 frames per second, which can be adjusted from 15 to 60 frames per second depending on the rate of change of the component's surface morphology. The exposure time is set to 200 microseconds, which can be optimized and adjusted from 100 to 500 microseconds depending on the ambient light intensity. The image output resolution is set to 1920×1080 pixels, which can be adapted from 1280×720 to 4096×2160 depending on the detection accuracy requirements. The piezoelectric acoustic emission sensor is set to a sampling frequency of 2 MHz, which can be adjusted from 1 MHz to 5 MHz depending on the upper limit of the expected signal frequency generated by the defect. The signal acquisition threshold is set to 40 dB, and the analog signal amplification factor is set to 60 times. The amplification factor can be selected within the range of 40 times to 100 times according to the actual signal strength level. After receiving the synchronization pulse, both types of devices synchronously start the acquisition operation, and the devices continuously output a continuous high-definition image sequence of the surface and continuous micro-vibration waveform data.

[0019] The acquired image and vibration waveform data are then transmitted in real-time via gigabit Ethernet to a local storage unit for caching and retention. Image data is stored in uncompressed or low-compressed format to preserve complete texture details, while micro-vibration waveform data is stored continuously in binary format to ensure no loss of sampling accuracy. The storage unit synchronously records the original time stamp of each data reception and employs a data frame verification mechanism to ensure no data loss or misalignment during transmission. The local storage unit is configured with a redundant disk array to ensure the integrity and security of the acquired data during storage.

[0020] Finally, a unified time reference is established based on the high-definition image sequence of the target area of ​​the steel structure acquired simultaneously and the microseismic signal of the rigid connection, so as to achieve precise alignment between each frame of the image and the time axis of the microseismic signal. The specific implementation process of this step will be described in detail later.

[0021] The security benchmark comparison module 20 is used to perform texture structure analysis on the high-definition image sequence of the surface, generate a set of visual features representing the integrity of the surface, and compare the set of visual features with a preset security benchmark.

[0022] Specifically, the high-definition image sequence of the steel structure is first grayscaled, then local windows are divided and the gray-level co-occurrence matrix is ​​calculated. Three types of features, namely edge continuity, gray-level uniformity, and texture direction, are extracted and combined to obtain a set of visual features that characterize the surface state of the component. The specific implementation process of this step will be described in detail later.

[0023] The pre-defined safety benchmark is obtained through pre-calibration using non-destructive standard steel structure specimens. Standard specimens, identical in material, processing technology, and surface treatment to the components under test and confirmed to be free of surface defects by flaw detection, are selected. Twenty sets of high-resolution image sequences, each containing 100 frames, are captured under identical acquisition parameters and lighting conditions as in the field. Multiple sets of standard visual feature sets are generated using the same grayscale conversion, grayscale co-occurrence matrix calculation, and texture feature extraction processes. The average value of each dimension's index within these multiple standard feature sets is used as the benchmark center value. The upper and lower boundaries of the safety benchmark interval are constructed using the average value ± 3 times the standard deviation. Specifically, the arithmetic mean and standard deviation of the index values ​​for the same dimension in the visual feature sets of all standard specimens are calculated. The lower boundary of this dimension is set as the average value minus 3 times the standard deviation, and the upper boundary is set as the average value plus 3 times the standard deviation. Each dimension is calculated in this way and then combined to form a complete safety benchmark interval. The choice of 3 times the standard deviation ensures that over 99.7% of the data in the normal samples fall within this interval, effectively balancing the false positive and false negative rates. If the on-site environment fluctuates significantly, the discrimination scale can be tightened by using the average value ± 2 standard deviations, or the discrimination scale can be relaxed by directly using the minimum and maximum values ​​of all samples as the boundary. The above alternatives can be flexibly selected according to the actual detection accuracy requirements.

[0024] Finally, the indicators of each dimension of the real-time visual feature set are compared with the pre-calibrated safety benchmark interval to determine whether all indicators of the real-time features fall within the benchmark interval, thereby determining whether there is visible damage on the surface of the steel structure.

[0025] The microseismic activity threshold comparison module 30 is used to perform time-frequency transformation processing on the microseismic signal, extract a set of microseismic features that characterize the stress release properties inside the material, and compare the set of microseismic features with a preset microseismic activity threshold.

[0026] Specifically, the background vibration component is separated from the microseismic signal in the frequency domain to obtain the intrinsic vibration component. Three types of features, namely high-frequency bursts, energy attenuation, and dominant frequency drift, are extracted from the intrinsic vibration component and combined to generate a set of microseismic features that characterize the internal stress release properties of steel. The specific implementation process of this step will also be described in detail later.

[0027] The calibration of microseismic activity thresholds was completed in advance. This calibration process was uniformly carried out on-site using non-destructive steel structure samples to ensure that the calibration environment, equipment parameters, signal acquisition, and feature extraction procedures were completely consistent with the actual testing process. Standard steel structure samples, confirmed by professional flaw detection to be free of internal damage and structural defects, were selected. Under the same installation conditions, environmental conditions, and sampling parameters as the actual testing, 50 sets of standard microseismic signal data, each lasting 60 seconds, were continuously collected. For each set of signals, a complete process of signal preprocessing, intrinsic vibration component separation, and extraction of three types of microseismic features was sequentially performed, resulting in a set of 50 sets of microseismic features under standard non-destructive conditions. For the three dimensions of high-frequency burst characteristics, energy attenuation characteristics, and dominant frequency drift characteristics, the arithmetic mean and standard deviation of the corresponding feature values ​​for all standard samples were calculated. The result of superimposing three times the standard deviation of the mean value for each dimension was used as the microseismic activity threshold for that dimension, forming a complete set of microseismic activity thresholds containing parameters of all three dimensions. This calibration method can cover more than 99.7% of normal micro-vibration fluctuation data under non-destructive conditions, effectively avoiding misjudgments caused by environmental interference and equipment errors. If the on-site environmental interference fluctuations are large, the threshold standard can be tightened by superimposing the average value by 2 times the standard deviation to adapt to different testing conditions.

[0028] The real-time detected microseismic feature set is then compared dimension by dimension with the pre-calibrated microseismic activity threshold set to determine whether the real-time feature values ​​of each dimension exceed the corresponding microseismic activity threshold. When any feature value exceeds the threshold of that dimension, it is determined that the microseismic feature set exceeds the pre-calibrated microseismic activity threshold, and abnormal stress release is identified inside the steel structure.

[0029] In actual testing, when only one dimension exceeds its corresponding threshold while the other two dimensions remain within the normal range, it indicates a specific type of abnormal stress release activity within the component, but the scope and intensity of the anomaly are relatively limited. When two dimensions simultaneously exceed their respective thresholds, it indicates that the scope and intensity of the abnormal stress release activity have expanded, and the internal damage state of the material is more severe than when only one dimension is abnormal. When all three dimensions exceed their respective thresholds, it indicates that the stress release behavior within the component exhibits comprehensive anomalies in all three dimensions: suddenness, energy attenuation characteristics, and frequency drift characteristics. This situation corresponds to the highest damage risk level. Based on the combination of abnormal dimensions, the corresponding risk level can be appended to the output warning information, allowing subsequent testing personnel to take differentiated re-inspection confirmation or maintenance measures based on different combinations of anomalies.

[0030] By standardizing and collecting multi-dimensional microseismic features, and relying on sufficient non-destructive samples to calibrate adaptive thresholds to complete feature comparison, it is possible to accurately distinguish between normal environmental microseismic fluctuations of steel structures and abnormal microseismic activities caused by structural damage, thereby achieving quantitative identification and judgment of potential internal damage hazards in components.

[0031] The feature deviation calculation module 40 is used to map the visual feature set and the microseismic feature set to the same multidimensional space, construct a coupled feature group, and calculate the deviation degree of the two types of features in the coupled feature group.

[0032] Specifically, the sub-features of the visual feature set and the microseismic feature set are mapped to multi-dimensional spatial coordinate axes to obtain two sets of coordinate points, which are then superimposed to construct a coupled feature group. The spatial distance between the two sets of points is then calculated as the degree of deviation between the two types of features. The specific implementation process of this step will be described in detail later.

[0033] The latent fatigue defect determination module 50 is used to determine that there is a latent fatigue defect in the target area when the visual feature set is within the preset safety benchmark range, while the microseismic feature set exceeds the preset microseismic activity threshold and the deviation is greater than the preset risk threshold.

[0034] Specifically, the steps involve verifying that all indicators of the visual feature set are within the preset safety baseline, whether the high-frequency sudden features in the microseismic feature set continuously exceed the preset microseismic activity threshold, and whether the deviation is greater than the preset risk threshold. If all three conditions are met, it is determined that there is a hidden fatigue defect in the target area. The specific implementation process of this step will also be described in detail later.

[0035] The early warning information output module 60 is used to assess the reduction of the remaining life of the target area based on the changing trend of energy accumulation in the microseismic feature set, and output early warning information.

[0036] Specifically, the conversion relationship between microseismic energy and fatigue damage is pre-defined. The material's SN curve reflects the number of cycles required for the specimen to reach fatigue failure at different stress amplitude levels. This curve is pre-determined and plotted through cyclic loading tests of standard specimens at multiple stress amplitude levels. The Miner linear cumulative damage criterion assumes that the fatigue damage caused by cyclic loading at each stress amplitude level can be linearly accumulated. The specific calculation method is to divide the number of cycles for each loading level by the number of failure cycles corresponding to the SN curve at that stress amplitude level to obtain the cycle ratio for that loading level. The cycle ratios of all loading levels are then summed to obtain the current accumulated total fatigue damage of the specimen.

[0037] Next, a standard specimen of the same material as the steel structure to be tested was selected, and a graded cyclic loading test was carried out on a material fatigue testing machine. During the loading process, a micro-vibration sensor consistent with that used in the field was used to synchronously collect signals. In this embodiment, the fatigue test example uses 5 stress amplitude levels, with each stress amplitude being 40%, 50%, 60%, 70%, and 80% of the material's yield strength. The number of cycles set under each stress amplitude level is 10%, 25%, 50%, 75%, and 90% of the failure cycle number under the corresponding stress amplitude on the material's SN curve, forming a total of 25 loading condition points.

[0038] After each loading stage, the test is paused. Based on the stress amplitude and actual number of cycles of that loading stage, combined with the material's SN curve and Miner's linear cumulative damage criterion, the fatigue damage degree of the current specimen is calculated. Simultaneously, the cumulative energy value of all micro-vibration pulses within that loading stage is statistically analyzed. The energy value of each energy surge waveform segment is calculated by summing the squares of the amplitudes of all sampling points within that segment. The cumulative energy value of all micro-vibration pulses within a single loading stage is obtained by summing the energy values ​​of all pulse segments within that stage. Three sets of parallel specimen tests are repeated at each working point. All specimens are compiled to obtain 75 sets of fatigue damage degree and corresponding cumulative energy data. A linear fitting method is used to obtain the micro-vibration energy conversion factor corresponding to unit fatigue damage, and this factor is solidified as a universal conversion benchmark from energy to damage. Using three sets of parallel specimens and taking the average of the fitted parameters effectively eliminates the influence of random errors from a single test on the accuracy of the conversion factor calibration, ensuring the representativeness of the calibration results.

[0039] Next, during the remaining service life assessment, historical microseismic energy accumulation data for the entire service life of the target area is retrieved and superimposed with the microseismic energy values ​​for each time unit obtained from the current monitoring period to generate an energy accumulation time series covering the entire service period. The least squares method is used to linearly fit this time series to obtain the slope of the energy accumulation trend, which represents the microseismic energy accumulation rate per unit time. The total accumulated energy over the entire period is divided by a pre-calibrated energy damage conversion factor to obtain the current accumulated total fatigue damage in the target area. Combining the trend slope and the conversion factor, the damage increment per unit time is derived. The remaining allowable damage is obtained by subtracting the current total damage from the material's allowable total fatigue damage threshold. The remaining allowable damage is then divided by the damage increment per unit time to calculate the remaining service life of the target area. Finally, the difference between the component's design reference life and the remaining service life is divided by the design reference life to obtain the remaining service life reduction ratio, thus completing the quantitative assessment of the remaining service life reduction.

[0040] Finally, different warning levels are determined based on the remaining service life reduction ratio, as shown in Table 1. Warning information including the target area location identifier, defect type determination result, remaining service life reduction ratio, and estimated remaining service time is generated and pushed to the detection management terminal to prompt maintenance personnel to conduct targeted review and maintenance work.

[0041] Table 1: Classification of Warning Levels for Reduced Remaining Service of Steel Structures Warning Level Remaining life reduction ratio range Warning Explanation Mild warning Reduction ratio < 10% The remaining service life reduction in the target area is relatively small, and the overall service condition of the steel structure is safe. It is recommended that it be included in the routine inspection scope. Moderate warning 10% ≤ Reduction Rate ≤ 30% The remaining service life of the target area has been reduced to a moderate degree, and the steel structure faces a certain risk of fatigue damage. It is recommended to shorten the inspection cycle and conduct a special review. Severe warning Reduction ratio > 30% The remaining service life of the target area has been significantly reduced, and the risk of fatigue damage to the steel structure is high. It is recommended to immediately conduct a special inspection and develop an operation and maintenance plan. By calibrating the energy damage conversion benchmark through sample testing, quantifying fatigue damage and assessing the remaining service life reduction by combining energy accumulation trends, and then relying on the finite element digital model to realize damage parameter feedback and service life recalculation, a complete closed-loop control of defect identification, service life warning and dynamic model update is formed, which effectively improves the accuracy and dynamic response capability of steel structure service safety assessment.

[0042] In one possible implementation, the image signal acquisition module 10 further includes: Establish a unified time reference to ensure that each frame in the high-definition surface image sequence is strictly aligned with the time axis of the microseismic signal.

[0043] Specifically, after data acquisition is completed, a time reference unification operation is performed. The standard timestamp output from the high-precision global clock source of the synchronization signal generator is retrieved as the global unified timing basis. The acquisition time marker inherent in each image frame is extracted, and the acquisition time markers corresponding to the micro-seismic waveform segments are extracted synchronously. Using 1 millisecond as the minimum time sequence calibration unit, the image frames and the corresponding micro-seismic waveforms for each time period are matched and calibrated on a time axis. Since the time interval between each adjacent frame of the industrial area scan camera in the aforementioned example is approximately 33.3 milliseconds when sampling at 30 frames per second, and the piezoelectric acoustic emission sensor contains 2000 waveform sampling points per millisecond when sampling at 2 MHz, the following correspondence method is used when matching the image frames and micro-seismic waveforms in 1-millisecond units: For the 33.3-millisecond time period corresponding to each image frame, the complete waveform data of all micro-seismic waveform sampling points within that time period is used as the micro-seismic information associated with that frame, rather than extracting a single representative value. This fully preserves the time and frequency domain information of the micro-seismic waveform for subsequent feature extraction. To address the fixed timing difference caused by the camera's 30 frames per second and the sensor's 2 MHz sampling frequency, a linear interpolation algorithm is used to fill the time gap between the two types of data to complete the deviation compensation, ensuring that the time deviation between the exposure start time of the image frame and the start point of the corresponding micro-vibration waveform segment is controlled within ±0.5 milliseconds.

[0044] After time stamp matching and timing deviation compensation are performed on all image frames and corresponding microseismic waveforms, a one-to-one corresponding time-series data set is formed. Each data set contains a complete image frame and a continuous microseismic waveform segment that is strictly synchronized with the exposure time of that image frame, and the duration of the waveform segment is consistent with the time interval between two adjacent image frames. This operation eliminates the timing offset between image data and vibration signals, completes the spatiotemporal alignment of the two types of detection data, and obtains corresponding image information and microseismic vibration information at the same spatial location and the same load application time. Based on this, synchronous comparative analysis of component surface condition and internal material response data is achieved.

[0045] In one possible implementation, the security benchmark comparison module 20 further includes: The high-resolution surface image sequence is converted to grayscale to obtain a grayscale image. Multiple local windows are selected on the grayscale image, and the pixel grayscale co-occurrence matrix within each local window is calculated. Based on the grayscale co-occurrence matrix, texture structure analysis is performed to extract the edge continuity features, grayscale distribution uniformity features, and micro-texture direction features of the steel structure surface in the high-resolution surface image sequence. The edge continuity features, grayscale distribution uniformity features, and micro-texture direction features are combined to generate a visual feature set.

[0046] Specifically, the acquired high-resolution surface image sequence is first input into the image processing unit. A weighted average grayscale conversion algorithm is then used to perform grayscale conversion on each color image. For example, the pixel weights for the red channel are 0.299, the green channel is 0.587, and the blue channel is 0.114 to calculate the grayscale value pixel by pixel. After all pixels have been traversed, a single-channel grayscale image is output to remove color information interference and reduce the amount of data computation required for subsequent texture calculations.

[0047] For each grayscale image, a square local window is divided, with a uniform window side length of 16 pixels. The window slides across the entire image horizontally and vertically with a step size of 8 pixels, overlapping by 8 pixels horizontally and vertically between adjacent windows. This overlapping sliding method increases the window sampling density and improves the sufficiency of texture feature extraction. If the surface texture of the component to be detected has good uniformity and higher computational efficiency is required, a non-overlapping window division method with a step size of 16 pixels can also be used, ensuring that adjacent windows are exactly adjacent without overlapping. This can be chosen based on the combined requirements of detection accuracy and computational efficiency. When a window cannot form a complete 16-pixel window when sliding to the image edge, the missing area at the image boundary is filled with a grayscale value of 0 to ensure that all local windows involved in the calculation have a uniform size. The purpose of using a grayscale value of 0 for boundary filling is to maintain the uniformity of all local window sizes to facilitate matrix operations. When processing windows near the image edge, the low grayscale value introduced by the filling area has a limited impact on the calculation of texture indicators, which can be ignored when the total number of windows is large.

[0048] If the detection area needs to focus on the texture state of the component boundary, a mirror filling method can be used instead of filling with grayscale values ​​of 0. That is, the pixels inside the boundary are mapped to the outer filling area with the boundary pixels as the axis of symmetry, reducing the interference of filling values ​​on texture calculation. During the traversal, the pixel matrix corresponding to each local window is recorded, and the grayscale co-occurrence matrix is ​​calculated for all local windows. The pixel grayscale level is set to 64 levels during the calculation, and four offset directions are selected: horizontal, vertical, 45-degree, and 135-degree. The offset step size is 1 pixel distance. The co-occurrence frequency of different grayscale pixels in the corresponding directions within the window is counted to generate a two-dimensional pixel grayscale co-occurrence matrix corresponding to each local window.

[0049] Subsequently, texture structure analysis calculations are performed based on the pixel gray-level co-occurrence matrices generated within each local window. Specifically: all rows and columns of the gray-level co-occurrence matrix are traversed, and the matrix contrast value is obtained by summing the product of the difference between the matrix row and column coordinates and the corresponding element; the matrix entropy value is obtained by taking the negative of the sum of the products of the values ​​of each element and the logarithms of the elements; the matrix correlation value is obtained by calculating the mean and standard deviation of the gray levels of the matrix rows and columns and then performing a normalization operation based on the covariance of the matrix elements. Based on the element values ​​within this co-occurrence matrix, three types of texture indices are solved: the matrix contrast value characterizes the continuity of the steel structure surface edges, the matrix entropy value characterizes the uniformity of gray-level distribution, and the matrix correlation value characterizes the micro-texture orientation.

[0050] After calculating the three texture indices for each local window, the arithmetic mean of the contrast indices of all local windows within a single grayscale image is taken to obtain the edge continuity characterization value of that frame. The arithmetic mean of the entropy values ​​of all local windows is taken to obtain the grayscale distribution uniformity characterization value of that frame. The arithmetic mean of the correlation indices of all local windows is taken to obtain the micro-texture direction characterization value of that frame. The above three means are combined in a fixed order to form the three-dimensional visual feature vector of that frame. Finally, the values ​​of the three types of indices corresponding to all local windows of a single grayscale image are extracted sequentially and summarized to obtain the complete texture index data of a single frame image.

[0051] Finally, the edge continuity index, grayscale distribution uniformity index, and micro-texture direction index extracted from all frames of the same image sequence are integrated and combined in a fixed-dimensional order to form a visual feature set with fixed dimensions. Each set of data in the set corresponds to the surface integrity state representation information of the detection area at the same time.

[0052] By employing a complete image processing and feature comparison process, and relying on standardized grayscale conversion and grayscale co-occurrence matrix texture calculation methods, the surface condition of steel structures is quantitatively characterized. By relying on sufficient non-destructive standard samples to calibrate safety benchmarks, surface defects are automatically identified, effectively avoiding missed detections and subjective judgment errors caused by visual observation of a single image.

[0053] In one possible implementation, the microseismic activity threshold comparison module 30 further includes: The microseismic signal is decomposed in the frequency domain to separate the background vibration component caused by environmental interference and obtain the intrinsic vibration component caused by changes in the internal structure of the material. High-frequency burst features, energy attenuation features and dominant frequency drift features are extracted from the intrinsic vibration components. The high-frequency burst features, energy attenuation features and dominant frequency drift features are combined to generate a microseismic feature set.

[0054] Specifically, for the microseismic signals acquired in the aforementioned steps, i.e., the original microseismic time-domain signals, the signal processing unit first performs a DC bias removal operation on the original microseismic time-domain signals. The arithmetic mean of the entire signal segment is calculated and subtracted from all sampling points to bring the signal mean to zero, eliminating frequency domain analysis interference caused by DC offset in the acquisition link. After DC bias removal, the signal is segmented at fixed intervals of 10 milliseconds. An example of a single signal segment contains 20,000 discrete sampling points. Each segment is continuously truncated into a 2048-point window and subjected to a 2048-point Fast Fourier Transform. Subsequent transforms are performed with a fixed sliding step size, continuously truncating new 2048-point windows until all 20,000 sampling points of the segment have been processed, ensuring that the spectral information of the entire signal is fully utilized without data loss. A 2048-point frequency domain complex array is output, where the real and imaginary parts of the complex numbers correspond to the in-phase and quadrature amplitude components of the frequency points, respectively. The transformation window can be truncated using either a non-overlapping truncation method with a step size of 2048 sampling points, where adjacent windows are exactly adjacent without overlapping; or an overlapping truncation method with a step size of 1024 sampling points, where adjacent windows overlap by 50%. The choice between these two methods depends on the characteristics of the signal at the time. If a segment has fewer sampling points than a complete window, it will be concatenated with the next signal segment to accumulate 2048 sampling points before the transformation continues.

[0055] Next, environmental interference data was collected under no-load and no component deformation conditions. Each acquisition lasted 60 seconds, and 30 sets of data were collected repeatedly. Each set of data was processed using the same segmentation and fast Fourier transform process as the real-time microseismic signal. A single set of frequency domain amplitude curves was generated from the complex modulus values ​​of each frequency point. The arithmetic mean of the 30 sets of amplitudes at the same frequency point was calculated to generate the reference spectrum of environmental vibration amplitude distribution.

[0056] The magnitude values ​​of each frequency point in the real-time signal frequency domain complex array are used as the real-time amplitude. The difference between these amplitude values ​​and the corresponding frequency point amplitude values ​​in the environmental vibration amplitude distribution reference spectrum is calculated. A globally fixed amplitude difference threshold is set to 0.05 volts. A difference greater than 0.05 volts between the real-time amplitude and the reference amplitude is considered a valid component; otherwise, it is considered a background vibration component. In case of field interference or sensor model changes, an adaptive threshold of 2 to 3 times the standard deviation of the reference amplitude at each frequency point can be used to complete the replacement.

[0057] Next, a 2048-point frequency domain complex array is constructed for the inverse transform input. The array satisfies the conjugate symmetry constraint of positive and negative frequencies. For frequencies determined to be effective components, the original complex numbers are directly copied and filled into the corresponding positions in the array. The real and imaginary parts of the complex numbers at the background components and frequency vacancy positions are uniformly set to zero, simultaneously completing the conjugate matching of positive and negative frequencies, ensuring that the inverse transform output is a real time-domain waveform. After the array is constructed, a 2048-point inverse fast Fourier transform program is input to perform waveform reconstruction, and the final output is the intrinsic vibration component generated only by the internal structural changes of the steel structure.

[0058] Next, we identify the frequency of occurrence of short-term energy surge waveform segments of intrinsic vibration components as high-frequency burst features, calculate the waveform amplitude decay rate as energy decay features, and track the frequency center shift amplitude in the time-frequency domain as the dominant frequency drift feature. The specific acquisition process of these three features in this step will be explained in detail later.

[0059] Finally, the high-frequency burst characteristic values, energy attenuation characteristic values, and dominant frequency drift characteristic values ​​calculated within the same detection cycle are arranged, integrated, and bound in a fixed dimensional order to construct a microseismic characteristic set with fixed dimensions and unique parameters. The three sets of characteristic parameters within the set synchronously correspond to the internal stress activity state of the steel structure during the same detection period, thereby achieving unified quantitative collection of multi-dimensional microseismic damage characteristics.

[0060] In one possible implementation, the microseismic activity threshold comparison module 30 further includes: Identify waveform segments with short-term energy surges in the intrinsic vibration component and use the frequency of occurrence of the waveform segments as high-frequency burst characteristics; calculate the amplitude attenuation rate of the intrinsic vibration component during propagation and use the amplitude attenuation rate as energy attenuation characteristics; track the frequency center migration trajectory of the intrinsic vibration component in the time-frequency domain and use the change amplitude of the frequency center migration trajectory as the dominant frequency drift characteristics.

[0061] Specifically, the separated intrinsic vibration components are first input into the feature extraction unit. Short-time energy windowing is then performed on the continuous intrinsic vibration time-domain waveforms. The sliding window length is set to 2 milliseconds, and the sliding step size is 0.5 milliseconds. When a window cannot form a complete window at the beginning or end of the waveform, the incomplete window is discarded and not included in the short-time energy calculation. The sum of the squares of the amplitudes of all sampling points within the window is calculated window by window as the short-time energy value for a single window, thus obtaining a continuously distributed short-time energy sequence along the time axis. The short-time energy threshold is pre-calibrated under no-load conditions. Thirty sets of 60-second intrinsic vibration baseline signals are collected under no-load and no component deformation conditions. Multiple sets of short-time energy values ​​are obtained using a completely consistent windowing calculation process. Under normal conditions, the mean of all values ​​plus three times the standard deviation is taken as a fixed short-time energy threshold. When the interference in the field environment is weak, the mean plus two times the standard deviation can also be used to complete the threshold calibration.

[0062] If the short-term energy value of a single window exceeds the threshold, the corresponding time window coverage area is determined to be a waveform segment with a sudden energy increase. When adjacent time windows continuously exceed the threshold, the adjacent windows are merged into a single continuous waveform segment, with a unified definition of the start and end times. The interval between two waveform segments must be at least the length of a complete time window; adjacent segments with an interval of less than 2 milliseconds are considered the same continuous event and are merged. Based on the above merging rules, for each identified independent waveform segment, subsequent amplitude decay rate and dominant frequency drift characteristics are calculated. After traversing all valid time windows, the total number of independent segments exceeding the threshold is counted, and this count represents the high-frequency burst characteristic.

[0063] Next, complete time-domain waveforms are extracted from each of the identified independent waveform segments. The first peak sampling point of a single waveform segment is taken as the starting time of energy propagation. The amplitude values ​​corresponding to the sampling points are extracted sequentially at fixed time intervals of 0.1 milliseconds. The least squares method is used to perform linear fitting on multiple sets of time and amplitude corresponding data. The absolute value of the slope of the fitted line is taken as the amplitude decay rate of a single waveform segment. The arithmetic mean of the decay rates corresponding to all segments is calculated to obtain the energy decay characteristics corresponding to the complete intrinsic vibration components.

[0064] Next, the main frequency drift feature extraction operation is performed on each independent waveform segment. For each independent waveform segment, sampling point positions are extracted sequentially starting from the segment's beginning time at fixed time intervals of 0.1 seconds. At each extraction time position, local waveform data of 1 milliseconds before and after that time are taken to form a local signal segment with a total length of 2 milliseconds. A 2048-point Fast Fourier Transform is performed to obtain the frequency domain data of this local segment. If the extraction position is close to the beginning or end of the segment, resulting in insufficient effective data length, the missing positions are padded with zeros to reach 2048 points to ensure the normal execution of the transformation operation. Using the amplitude of each frequency point within the local frequency domain as a weighting coefficient, the frequency center value corresponding to the extraction time is calculated by multiplying each frequency point value by its corresponding amplitude, summing the results, and then dividing by the sum of all frequency point amplitudes. The data extraction, frequency domain transformation, and frequency center calculation are repeated at fixed intervals along the waveform time axis to obtain frequency center values ​​corresponding to multiple times. These values ​​are then integrated in chronological order to form the segment's frequency center migration trajectory. Find the maximum and minimum values ​​of the frequency center values ​​within the trajectory, and use the difference between the two as the main frequency drift amplitude of a single waveform segment. Calculate the arithmetic mean of the drift amplitudes of all segments, distribute the weights of each segment evenly, and finally obtain the main frequency drift characteristics.

[0065] If no energy surge waveform segment is identified in the intrinsic vibration components, the values ​​of high-frequency burst feature, energy attenuation feature, and dominant frequency drift feature are all set to zero, indicating that there is no significant stress release event in the current period and the internal state of the component is stable. If only a single waveform segment is identified, the attenuation rate and dominant frequency drift amplitude calculated from that segment are directly used as the corresponding feature values ​​to complete the feature parameter output. The above settings can ensure the stable closed-loop execution of the feature extraction process under special working conditions such as no event or single event.

[0066] Finally, the high-frequency burst characteristic values, energy attenuation characteristic values, and dominant frequency drift characteristic values ​​calculated within the same detection cycle are integrated and stored in a fixed order. The three sets of values ​​correspond one-to-one with the stress release state inside the component during the same period, forming a complete and unified set of microseismic characteristics.

[0067] Three types of microseismic indices are quantified using three general signal processing methods: standardized short-time energy window detection, fixed-interval sampling least-squares fitting, and amplitude-weighted solution of center frequency. This fully extracts quantitative features that reflect the degree of stress release within the steel structure, providing quantitative data support for subsequent comprehensive judgment of component defects by combining surface visual features.

[0068] In one possible implementation, such as Figure 2 As shown, the feature deviation calculation module 40 further includes: Each coordinate axis in the multidimensional space corresponds to a sub-feature in the visual feature set and a sub-feature in the microseismic feature set, respectively. The sub-features in the visual feature set are mapped to a set of visual feature coordinate points in the multidimensional space; the sub-features in the microseismic feature set are mapped to a set of microseismic feature coordinate points in the multidimensional space; the visual feature coordinate points set and the microseismic feature coordinate points set are superimposed to construct a coupled feature group; based on the coupled feature group, the spatial distance between the visual feature coordinate points set and the microseismic feature coordinate points set is calculated, and this spatial distance is used as the degree of deviation between the two types of features.

[0069] Specifically, a multidimensional space for feature fusion, namely a multidimensional Euclidean space, is constructed. The total number of spatial dimensions is equal to the sum of the number of sub-features in the visual feature set and the number of sub-features in the microseismic feature set. In the aforementioned steps, the visual features contain 3 sub-features, and the microseismic features contain 3 sub-features. Therefore, 6 independent coordinate axes are set, and each coordinate axis corresponds in a fixed order to edge continuity features, gray-level distribution uniformity features, micro-texture direction features, high-frequency burst features, energy attenuation features, and dominant frequency drift features. All sub-feature values ​​are uniformly normalized to min-max before mapping. The global maximum and global minimum values ​​used for normalization are taken from the same batch of non-destructive sample feature data collected during the standard sample calibration stage. This batch of samples is completely consistent with the samples used for visual and microseismic benchmark calibration. The global maximum and global minimum values ​​corresponding to each sub-feature are calculated separately. The calculation method is to subtract the minimum value from the current feature value and divide by the difference between the maximum and minimum values ​​to uniformly map the values ​​of all sub-features to the range of 0 to 1, eliminating the dimensional differences and numerical scale differences between different features and ensuring the consistency of coordinate operations in the multidimensional space.

[0070] Next, a single detection cycle is divided into several time units of equal length at fixed time intervals, with each time unit length set to 100 milliseconds. All time units are arranged continuously along the time axis, serving as common alignment nodes for visual features and microseismic features. After normalization preprocessing, a spatial mapping operation for visual features is performed. For each time unit, the normalized values ​​of three visual sub-features corresponding to all image frames within that time range are extracted. The arithmetic mean of each sub-feature is calculated, and these three means are assigned to the three coordinate axes of the corresponding visual features in the multidimensional space. The three coordinate axes corresponding to the microseismic features in the space are uniformly set to 0 as placeholders. The 0 value only serves as a placeholder to indicate that this dimension has not yet been assigned a valid value at the current stage and does not participate in subsequent coupling operations. It will be replaced by the valid feature values ​​of the corresponding dimension in the overlay step, thus obtaining the visual feature coordinate points corresponding to that time unit. All time units within the detection cycle are traversed, and the corresponding number of coordinate points are generated sequentially. All coordinate points are arranged in chronological order to form a set of visual feature coordinate points in the multidimensional space.

[0071] Similarly, the same mapping rules are used to process the microseismic feature set. For each time unit, the microseismic signal segments within the corresponding time period are extracted. The normalized values ​​of the three microseismic sub-features corresponding to each segment are assigned to the three coordinate axes of the corresponding microseismic feature in the multidimensional space. The three coordinate axes corresponding to the visual features in the space are uniformly set to 0 as a placeholder. The 0 value only serves as a placeholder to indicate that the dimension has not yet been assigned a valid value at the current stage and does not participate in subsequent coupling operations. It will be replaced by the valid feature values ​​of the corresponding dimension in the superposition step to obtain the microseismic feature coordinate points corresponding to the time unit. If a single time unit contains multiple microseismic signal segments, the arithmetic mean of the sub-feature values ​​of each segment is calculated before mapping. All time units within the detection period are traversed, and the corresponding number of coordinate points are generated sequentially. All coordinate points are arranged in chronological order to form a set of microseismic feature coordinate points in the multidimensional space.

[0072] Next, based on a unified time unit division, a unit-by-unit alignment and overlay is performed on the visual feature coordinate point set and the microseismic feature coordinate point set. First, the time intervals covered by both types of point sets are extracted, and only time units within these intervals are retained. Units at the beginning and end of non-overlapping time intervals are discarded and not included in the coupling operation. For each retained time unit, valid values ​​of the three visual dimensions are extracted from the visual coordinate points, and valid values ​​of the three microseismic dimensions are extracted from the microseismic coordinate points. These two types of valid values ​​are combined in coordinate axis order to replace the original occupant reference values ​​in the coordinate points, generating multi-dimensional coordinate points that simultaneously contain complete visual and microseismic dimensions. If one type of feature data is missing in a retained time unit, that unit is skipped and not included in the final coupled feature group. The complete coordinate points corresponding to all valid time units are combined in chronological order to form a coupled feature group that simultaneously carries surface state information and internal stress information.

[0073] Finally, the mean coordinate values ​​of the visual and microseismic feature point sets are obtained to obtain the centroids of the two types of features. The square root of the sum of the squares of the differences between the two centroids along their axes is calculated as the spatial distance characterizing the degree of deviation between the two types of features. The specific implementation process of this step will be described in detail later.

[0074] By unifying the dimensional normalization, aligning units of equal time, and filtering and superimposing common intervals, visual detection data and microseismic detection data from different sources are integrated into a unified multidimensional feature space, achieving standardized coupling of the two types of features and providing unified and repeatable data support for subsequent deviation calculation and comprehensive defect judgment.

[0075] In one possible implementation, the feature deviation calculation module 40 further includes: The average coordinates of all points in the visual feature coordinate point set are calculated to obtain the centroid of the visual feature. The average coordinates of all points in the microseismic feature coordinate point set are calculated to obtain the centroid of the microseismic feature. The sum of squares of the differences between the centroid of the visual feature and the centroid of the microseismic feature on each coordinate axis is calculated, and the square root of the sum of squares is taken to obtain the spatial distance. The spatial distance is then quantified as the degree of deviation.

[0076] Specifically, the visual feature coordinate point set consists of several multi-dimensional coordinate points generated in the preceding steps. Each coordinate point covers all visual feature coordinate axes and microseismic feature coordinate axes, with the microseismic coordinate axis dimension using a value of 0 as a placeholder. The set of all coordinate points in the visual feature coordinate point set is traversed. For each independent coordinate axis dimension, the coordinate values ​​of all points in that dimension are accumulated sequentially. After accumulation, the result is divided by the total number of coordinate points to obtain the centroid coordinate component corresponding to that dimension. After averaging all coordinate axis dimensions, the components of each dimension are combined in a fixed order according to the coordinate axes to form the complete visual feature centroid coordinate point, corresponding to the overall distribution center position of the visual feature set in multi-dimensional space.

[0077] Similarly, the same method of calculating the mean value along each dimension is used to process the microseismic feature coordinate point set. This set also consists of several multi-dimensional coordinate points generated in the previous steps. Each coordinate point covers all visual feature coordinate axes and microseismic feature coordinate axes, with the visual coordinate axis dimension using a value of 0 as a placeholder. All coordinate points are traversed to accumulate and average the values ​​of each dimension. The components of each dimension are combined in the same coordinate axis order to form the complete centroid coordinate point of the microseismic feature set, corresponding to the overall distribution center position of the microseismic feature set in multi-dimensional space. If any coordinate point set is empty and no valid coordinate points participate in the calculation, the deviation between the two types of features is directly recorded as zero, indicating that there is insufficient valid detection data to support the deviation determination in the current time period.

[0078] After calculating the centroid coordinates of the two types of features, the coordinate values ​​of the centroids of the visual features and the microseismic features are extracted dimension by dimension. The difference between the two centroid values ​​on the same coordinate axis is calculated, and the difference in each dimension is squared. The results of the squared operations in all dimensions are then summed to obtain the sum of the squared distances between the two types of centroids in multidimensional space. The arithmetic square root of the sum of squared distances is then calculated, and the resulting value is the Euclidean spatial distance between the set of coordinate points of the visual features and the set of coordinate points of the microseismic features. The larger the spatial distance value, the more significant the difference in the distribution of the two types of features and the higher the degree of deviation. This spatial distance value is directly quantified as the degree of deviation between the two types of features.

[0079] By solving the mean value of each dimension to determine the spatial distribution center of the two types of features, and combining the Euclidean distance calculation to quantify the overall distribution difference, a standardized quantitative calculation of the deviation between heterogeneous visual and microseismic features is achieved, providing a unified and comparable quantitative indicator for the comprehensive judgment of steel structure defects.

[0080] In one possible implementation, the latent fatigue defect determination module 50 further includes: Verify whether all indicators of the visual feature set fall within the preset safety benchmark range to confirm that there is no visible damage on the steel structure surface; verify whether the high-frequency sudden features in the microseismic feature set continuously exceed the preset microseismic activity threshold to confirm that there is abnormal stress release inside the material; verify whether the deviation is greater than the preset risk threshold to confirm that there is a logical conflict between the surface state and the internal state; when there is no visible damage on the surface, there is abnormal stress release inside and the logical conflict is established, it is determined that there is a hidden fatigue defect in the target area.

[0081] Specifically, the preset safety benchmark range is obtained by calibrating the same batch of non-destructive samples consistent with the previous feature normalization processing. For three visual sub-features—edge continuity, grayscale distribution uniformity, and microtexture direction—the arithmetic mean and standard deviation of the corresponding feature values ​​for all non-destructive samples are calculated. The interval between the average value of each dimension and three times the standard deviation is used as the safety benchmark range for the corresponding sub-feature. When conducting defect judgment, the visual sub-feature values ​​corresponding to all time units within the current inspection cycle are extracted, and the arithmetic mean of each is calculated as the current verification value. Each value is compared with the corresponding safety benchmark range. Values ​​falling within the interval or exactly equal to the interval boundary value are judged to fall within the safety benchmark range. When all sub-feature values ​​fall within the corresponding safety benchmark range, it is determined that there is no visible damage on the surface of the steel structure target area. If any sub-feature value exceeds the upper or lower limit of the interval, the judgment condition of no surface damage is not met.

[0082] The preset microseismic activity threshold is the high-frequency burst characteristic dimension threshold obtained from the calibration of the same batch of non-destructive samples. During verification, the high-frequency burst characteristic values ​​of each unit within the current detection cycle are read sequentially according to a uniformly divided time unit. Each unit is compared with the microseismic activity threshold. If the value is exactly equal to the threshold, it is determined that it has not exceeded the threshold. The number of time units that continuously exceed the threshold is counted. When the number of time units that continuously exceed the threshold reaches 3 or more, it is determined that the high-frequency burst characteristic continues to exceed the threshold, confirming that there is abnormal stress release inside the steel structure of the target area. If the number of consecutive exceedances is less than 3, it is determined that the internal stress release state is stable and does not meet the abnormal conditions.

[0083] The preset risk threshold is also based on the calibration of this batch of non-destructive samples. Multidimensional spatial mapping and centroid distance calculation are sequentially performed on all non-destructive samples to obtain a sample set of deviations between visual and microseismic features in the non-destructive state. The arithmetic mean and standard deviation of the sample set are statistically analyzed, and the result of the mean plus three times the standard deviation is used as the risk threshold. The deviation values ​​of the two types of features calculated in the current detection cycle are compared with the risk threshold. When the value is exactly equal to the risk threshold, the two types of features are considered to be matched. When the deviation value is greater than the risk threshold, it is determined that there is a logical conflict between the surface state and the internal state. When the deviation value is less than or equal to the risk threshold, the two types of features are considered to be matched and there is no logical conflict.

[0084] After independently verifying the above three conditions, the logical AND operation is performed to integrate the three verification results. Only when all three conditions are met simultaneously—no visible surface damage, abnormal internal stress release, and logical conflict between the surface and internal states—is a latent fatigue defect in the target area determined. If any one condition is not met, the target area is determined to be free of latent fatigue defects, and the corresponding output is a standard inspection conclusion indicating either normal steel structure condition or visible damage.

[0085] By verifying the surface visual condition, internal microseismic activity status and matching consistency with two types of features, and then using multi-condition logic and rules to complete a comprehensive judgment, the system can accurately identify early internal latent fatigue defects in steel structures with no obvious surface damage, effectively supplementing the shortcomings of single detection methods in identifying early internal damage.

[0086] In one possible implementation, the latent fatigue defect determination module 50 further includes: Based on the frequency of occurrence of high-frequency burst features in the microseismic feature set within a unit time, a predefined defect type library is matched; when the high-frequency burst feature is characterized by short-period dense pulses, the latent fatigue defect is determined to be the propagation of microcracks in the matrix; when the high-frequency burst feature is characterized by long-period sparse pulses, the latent fatigue defect is determined to be the degradation of the welded area.

[0087] Specifically, a defect type library is constructed beforehand, based on standard steel structure specimens with known defect types. For example, Q355 steel structure substrate specimens conforming to material standards are selected, and axial cyclic fatigue loading with a stress ratio of 0.1, a loading frequency of 10 Hz, and a stress amplitude of 180 MPa is applied to induce microcrack propagation in the substrate. During loading, the high-frequency burst pulse rate of the acoustic emission signal is monitored in real time. When the pulse rate enters a relatively stable stage (i.e., the coefficient of variation of the number of pulses per minute is less than 15% over 5 consecutive minutes), the material is considered to have entered a stable microcrack propagation stage, and calibration signals are collected at this point. Under sensor installation methods, signal sampling frequencies, and environmental conditions completely consistent with actual testing, 40 sets of micro-vibration signals, each 60 seconds long, are continuously collected. All signals correspond to the microcrack propagation state of the substrate confirmed by flaw detection. Steel structure specimens of the same material with standard butt welded joints are selected, and alternating cyclic aging loading with a stress ratio of 0.2, a loading frequency of 5 Hz, and a stress amplitude of 120 MPa is applied to induce degradation in the welded area. Using a pulse occurrence rate coefficient of variation of less than 15% as the marker for entering the stable degradation stage, 40 sets of microseismic signals, each 60 seconds long, were continuously collected under the same operating conditions. All signals corresponded to the degradation state of the welded area as confirmed by testing. Signal preprocessing and high-frequency burst feature extraction were uniformly performed on all microseismic signals from both types of samples. The occurrence time points of pulse events were statistically analyzed for each sample, and the average time interval between adjacent pulses and the pulse occurrence frequency per second within each sample were calculated.

[0088] Next, for the sample group with matrix microcrack propagation, the arithmetic mean and sample standard deviation of the average pulse interval parameter were calculated. A 95% confidence interval was constructed by multiplying the critical value of the t-distribution by the standard error, yielding the upper and lower boundaries of the interval parameter. The upper boundary was determined as the short-cycle judgment threshold; that is, when the average pulse interval is less than this threshold, it belongs to the short-cycle pulse interval. For the sample group with weld degradation, the same statistical method was used to construct a 95% confidence interval, yielding the upper and lower boundaries of the interval parameter. The lower boundary was determined as the long-cycle judgment threshold; that is, when the average pulse interval is greater than this threshold, it belongs to the long-cycle pulse interval. The pulse interval corresponding to matrix microcrack propagation is usually shorter, and its confidence interval is located to the left of the numerical axis; the pulse interval corresponding to weld degradation is usually longer, and its confidence interval is located to the right of the numerical axis. The numerical range between the short-cycle judgment threshold and the long-cycle judgment threshold is the transitional range for the interval parameter to be verified.

[0089] The classification thresholds for frequency occurrence per unit time were independently calibrated using the same statistical calculation method. A 95% confidence interval was constructed for the frequency occurrence per unit time parameter of the matrix microcrack propagation sample group, obtaining the upper and lower boundaries of this frequency parameter. The lower boundary was determined as the dense pulse judgment threshold; that is, when the frequency occurrence per unit time is greater than this threshold, it belongs to the dense pulse interval. Similarly, a 95% confidence interval was constructed for the frequency occurrence per unit time parameter of the weld area degradation sample group, obtaining the upper and lower boundaries of this frequency parameter. The upper boundary was determined as the sparse pulse judgment threshold; that is, when the frequency occurrence per unit time is less than this threshold, it belongs to the sparse pulse interval. The frequency confidence interval for the matrix microcrack propagation group is located to the right of the numerical axis, and the frequency confidence interval for the weld area degradation group is located to the left of the numerical axis. The numerical interval between the dense pulse judgment threshold and the sparse pulse judgment threshold is the transitional interval for the frequency parameter to be verified.

[0090] If the sample data does not satisfy the normal distribution assumption, the percentile method is used to construct the 95% confidence interval, with the 2.5th percentile as the lower boundary and the 97.5th percentile as the upper boundary. Short-cycle, long-cycle, dense pulse, and sparse pulse judgment thresholds are determined according to the corresponding rules mentioned above, while the value logic for adjacent boundaries remains unchanged. The defect type name is then bound to the corresponding short-cycle, long-cycle, dense pulse, and sparse pulse judgment thresholds for storage, forming a defect type library containing both types of defect patterns.

[0091] Once a latent fatigue defect is determined in the target area, the current detection cycle is defined as 300 seconds forward from that determination point. The time points of all high-frequency burst pulse events identified within this cycle are statistically analyzed and arranged chronologically to form a pulse time sequence. If the total number of pulse events identified within the 300-second detection cycle is less than 10, the statistical sample size is deemed insufficient, defect type matching is temporarily suspended, and a message indicating insufficient data and requesting an extension of the detection time is output. The total number of pulses within the detection cycle is counted, and the total number is divided by the total detection cycle duration to obtain the frequency of high-frequency burst features per unit time. The time interval between every two adjacent pulses in the time sequence is calculated sequentially, and the arithmetic mean of all interval values ​​is obtained to obtain the average pulse interval of the current detection cycle.

[0092] The calculated frequency of occurrence per unit time and average pulse interval are then compared item by item with the preset classification thresholds in the defect type library. When the average pulse interval is less than the short-cycle threshold and the frequency of occurrence per unit time is greater than the dense pulse threshold, the current high-frequency burst feature is determined to be short-cycle dense pulses, corresponding to the latent fatigue defect type of matrix microcrack propagation. When the average pulse interval is greater than the long-cycle threshold and the frequency of occurrence per unit time is less than the sparse pulse threshold, the current high-frequency burst feature is determined to be long-cycle sparse pulses, corresponding to the latent fatigue defect type of weld area degradation. If the parameter value falls within the middle range of the two thresholds, or exceeds the parameter coverage range of the two types of defect samples, a prompt result indicating that the defect type needs to be reviewed is output, ensuring a complete closed loop in the judgment process.

[0093] By statistically analyzing confidence intervals of known defect samples and defining four types of boundary thresholds, a standardized defect type library is constructed. Combined with the dual-parameter threshold comparison of real-time pulse frequency and interval, automatic defect type matching is achieved. This can further distinguish the specific location and mechanism of defect generation based on the identification of latent fatigue defects, thereby improving the precision of the detection results and their engineering guidance value.

[0094] In one possible implementation, the warning information output module 60 further includes: The location information of the target area identified as having latent fatigue defects and the corresponding set of microseismic features are fed back to the steel structure digital model; the stress parameters of the target area are updated in the steel structure digital model, and the remaining life prediction value of the target area is recalculated based on the updated stress parameters.

[0095] Specifically, the steel structure digital model is a full-size parametric structural simulation model built by those skilled in the art based on a general finite element analysis platform. The model contains the complete geometric topology, material constitutive parameters, boundary constraints, and design load cases of the target steel structure. Each structural unit within the model is bound to a unique location code, corresponding one-to-one with the location identifier of the target area detected on-site. After completing the latent fatigue defect determination and life assessment, the location identifier of the target area and the corresponding complete microseismic feature set are synchronously transmitted to the steel structure digital model. The location code is used to match and locate the corresponding structural unit within the model, achieving a precise mapping between on-site detection data and simulation model units.

[0096] Based on the total fatigue damage results corresponding to the microseismic feature set, the stress parameters of the target structural element are updated. The elastic modulus value of the target structural element is linearly reduced according to the ratio of fatigue damage degree to the total damage threshold. That is, the elastic modulus value used for stiffness matrix assembly in the current loading step is the original elastic modulus multiplied by one minus the ratio of fatigue damage degree to the total damage threshold, thus characterizing the degradation effect of fatigue cumulative damage on the element's bearing stiffness. This reduction process is only used for the mechanical response analysis of the element in subsequent simulation calculations and does not affect the intrinsic parameters of the material. It is only effective within the current detection cycle. In the next cycle, the reduction coefficient is recalculated and updated based on the measured damage degree. The reduction of the fatigue strength design value is used for allowable stress correction in the element fatigue life calculation. Specifically, the original fatigue strength design value is multiplied by (1 minus the ratio of fatigue damage degree to the total damage threshold) to obtain the reduced fatigue strength value. After the stress parameters are updated, the nominal stress fatigue life calculation module built into the model is called. Combining the design load spectrum and the updated material parameters, the remaining life prediction value of the corresponding structural unit in the target area is recalculated. The life distribution data of the corresponding area in the model is updated synchronously to complete the dynamic correction of the digital model.

[0097] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0098] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0099] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application intends to include such modifications and variations.

Claims

1. An intelligent detection system for steel structure defects based on image recognition, characterized in that, The system includes: The image signal acquisition module is used to simultaneously acquire high-definition image sequences of the target area and microseismic signals rigidly connected to the target area during the application of service loads to steel structure components. The security benchmark comparison module is used to perform texture structure analysis on the high-definition image sequence of the surface, generate a set of visual features representing the integrity of the surface, and compare the set of visual features with a preset security benchmark. The microseismic activity threshold comparison module is used to perform time-frequency transformation processing on the microseismic signal, extract a set of microseismic features that characterize the stress release properties inside the material, and compare the set of microseismic features with a preset microseismic activity threshold. The feature deviation calculation module is used to map the visual feature set and the microseismic feature set to the same multidimensional space, construct a coupled feature group, and calculate the deviation degree of the two types of features in the coupled feature group; The latent fatigue defect determination module is used to determine that there is a latent fatigue defect in the target area when the visual feature set is within the preset safety benchmark range, while the microseismic feature set exceeds the preset microseismic activity threshold, and the deviation is greater than the preset risk threshold. The early warning information output module is used to assess the reduction of the remaining life of the target area based on the changing trend of energy accumulation in the microseismic feature set, and output early warning information.

2. The intelligent steel structure defect detection system based on image recognition as described in claim 1, characterized in that, The synchronous acquisition of high-resolution surface image sequences of the target area and microseismic signals rigidly connected to the target area also includes: Establish a unified time reference to ensure that each frame in the high-definition surface image sequence is strictly aligned with the time axis of the microseismic signal.

3. The intelligent steel structure defect detection system based on image recognition as described in claim 1, characterized in that, The high-resolution image sequence of the surface is subjected to texture structure analysis to generate a set of visual features representing the integrity of the surface, including: The high-resolution image sequence of the surface is converted to grayscale to obtain a grayscale image; Select multiple local windows on the grayscale image and calculate the pixel grayscale co-occurrence matrix within each local window; Based on the gray-level co-occurrence matrix, texture structure analysis is performed to extract edge continuity features, gray-level distribution uniformity features, and micro-texture orientation features of the steel structure surface from the high-resolution surface image sequence. The edge continuity feature, grayscale distribution uniformity feature, and microtexture direction feature are combined to generate a visual feature set.

4. The intelligent steel structure defect detection system based on image recognition as described in claim 1, characterized in that, The microseismic signal is subjected to time-frequency transformation processing to extract a set of microseismic features characterizing the internal stress release properties of the material, including: The microseismic signal is decomposed in the frequency domain to separate the background vibration component caused by environmental interference and obtain the intrinsic vibration component caused by changes in the internal structure of the material. High-frequency burst characteristics, energy attenuation characteristics, and dominant frequency drift characteristics are extracted from the intrinsic vibration components. The high-frequency burst characteristics, energy attenuation characteristics, and dominant frequency drift characteristics are combined to generate a set of microseismic features.

5. The intelligent steel structure defect detection system based on image recognition as described in claim 4, characterized in that, Extracting high-frequency burst features, energy attenuation features, and dominant frequency drift features from the intrinsic vibration components includes: Identify waveform segments with short-term energy surges in the intrinsic vibration components, and use the frequency of occurrence of the waveform segments as high-frequency burst features; Calculate the amplitude decay rate of the intrinsic vibration component during propagation, and use the amplitude decay rate as an energy decay characteristic; The frequency center migration trajectory of the intrinsic vibration component in the time-frequency domain is tracked, and the change amplitude of the frequency center migration trajectory is used as the dominant frequency drift characteristic.

6. The intelligent steel structure defect detection system based on image recognition as described in claim 1, characterized in that, The visual feature set and the microseismic feature set are mapped to the same multidimensional space to construct a coupled feature group, and the degree of deviation between the two types of features in the coupled feature group is calculated, including: The coordinate axes of the multidimensional space correspond to the sub-features in the visual feature set and the sub-features in the microseismic feature set, respectively. Map each sub-feature in the visual feature set to a set of visual feature coordinate points in a multi-dimensional space; Map each sub-feature in the microseismic feature set to a set of microseismic feature coordinate points in a multi-dimensional space; By superimposing the visual feature coordinate point set and the microseismic feature coordinate point set, a coupled feature group is constructed; Based on the coupled feature set, the spatial distance between the visual feature coordinate point set and the microseismic feature coordinate point set is calculated, and the spatial distance is used as the degree of deviation between the two types of features.

7. The intelligent steel structure defect detection system based on image recognition as described in claim 6, characterized in that, Calculate the spatial distance between the set of visual feature coordinate points and the set of microseismic feature coordinate points, and use this spatial distance as the degree of deviation between the two types of features, including: Calculate the average coordinates of all points in the visual feature coordinate point set to obtain the centroid of the visual feature; Calculate the average coordinates of all points in the set of microseismic characteristic coordinate points to obtain the centroid of the microseismic feature. The sum of squares of the differences between the centroids of the visual features and the centroids of the microseismic features on each coordinate axis is calculated, and the square root of the sum of squares is taken to obtain the spatial distance. The spatial distance is then quantified as the degree of deviation.

8. The intelligent steel structure defect detection system based on image recognition as described in claim 1, characterized in that, The target area was determined to have latent fatigue defects, including: Verify that all indicators of the visual feature set fall within the preset safety benchmark range and confirm that there is no visible damage on the steel structure surface; Verify whether the high-frequency burst features in the microseismic feature set continuously exceed the preset microseismic activity threshold to confirm the existence of abnormal stress release inside the material; Verify whether the deviation exceeds a preset risk threshold to confirm that there is a logical conflict between the surface state and the internal state; When there is no visible damage on the surface, abnormal stress release inside, and a logical conflict is established, it is determined that there is a hidden fatigue defect in the target area.

9. The intelligent steel structure defect detection system based on image recognition as described in claim 1, characterized in that, After determining that there are latent fatigue defects in the target area, the following steps are also included: Based on the frequency of occurrence of high-frequency sudden features in the microseismic feature set per unit time, a predefined defect type library is matched. When the high-frequency burst characteristics manifest as short-period dense pulses, the latent fatigue defect is determined to be the propagation of microcracks in the matrix. When the high-frequency burst characteristics manifest as long-period sparse pulses, the latent fatigue defect is determined to be welding area degradation.

10. The intelligent steel structure defect detection system based on image recognition as described in claim 1, characterized in that, After outputting the warning information, it also includes: The location information of the target area identified as having hidden fatigue defects and the corresponding set of microseismic features are fed back to the digital model of the steel structure. Update the stress parameters of the target area in the digital model of the steel structure, and recalculate the predicted remaining life of the target area based on the updated stress parameters.

Images