Intelligent structural damage identification method based on visual modal multi-scale feature clustering

Abstract

The application provides a structural damage intelligent identification method based on visual modal multi-scale feature clustering, first, high frame rate digital video under a free vibration state of a structure is collected, and a phase optical flow visual algorithm is used to analyze the video, and high spatial resolution vibration responses of the structure are calculated; second, Ibrahim time domain method is used to identify modal parameters, and modal vibration modes of the structure are obtained; then, Gaussian multi-scale signal analysis theory is used to perform multi-scale characterization on the obtained vibration modes, and normalized vibration mode curvature energies under each scale are calculated; finally, all measuring points are taken as samples, normalized vibration mode curvature energies under all scales are taken as sample multi-dimensional features, K-Means clustering analysis under multi-scale features is performed, and outliers in a clustering result are determined as damage positions. The application can realize accurate positioning of damage positions and positions close to the damage positions in a noise environment without manual intervention.

Description

Technical Field

[0001] This invention belongs to the field of mechanical structure health monitoring technology, and specifically relates to an intelligent identification method for structural damage based on visual modality multi-scale feature clustering. Background Technology

[0002] Large-scale civil engineering structures and foundations are vital to national welfare and people's livelihoods, involving massive investments and high costs. Although their design lifespan is typically several decades or even centuries, during long-term operation, due to numerous adverse factors such as material aging, fatigue effects, and environmental loads, the structures inevitably experience wear, cracks, and voids, leading to a decrease in their strength and performance degradation, and even catastrophic accidents. Therefore, to reduce maintenance costs and ensure the safe service of the structures, structural health monitoring is necessary, especially the identification and location of existing damage.

[0003] Damage alters the dynamic characteristics of a structure; therefore, structural health monitoring primarily relies on vibration signals for damage identification. Currently, there are two main methods for monitoring vibration signals: contact and non-contact. For contact monitoring methods, the discrete arrangement of sensors results in low spatial resolution and introduces a mass load effect, limiting the accuracy of damage identification and location. For non-contact monitoring methods, such as the mainstream laser Doppler vibration sensor, while the low spatial resolution problem can be solved, the measurement cycle is long and the equipment is expensive. Furthermore, damage identification in actual structures is often performed under noisy conditions and generally lacks baseline data for reference. Under these circumstances, identifying and locating the damaged area, or even adjacent damaged areas, presents significant challenges. Summary of the Invention

[0004] To overcome the shortcomings of existing structural health monitoring methods and address issues such as low spatial resolution, difficulty in identification under noisy environments, and reliance on baseline data in current damage identification methods, this invention aims to propose an intelligent structural damage identification method based on visual modality multi-scale feature clustering. This method has advantages such as high spatial resolution, simultaneous identification of damage and adjacent damage areas, high damage localization accuracy, no need for baseline data reference, and good noise robustness.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for intelligent structural damage recognition based on visual modality multi-scale feature clustering includes the following steps:

[0007] Step 1: Acquire high frame rate digital video of the structure under free vibration state, select the pixels of the imaging area corresponding to the structure as measurement points, and use the phase optical flow algorithm for motion tracking, calculate the vibration response of the structure and assemble the measurement points to obtain the structural vibration information matrix.

[0008] Step 2: Using the Ibrahim time-domain method, construct a mathematical model of the characteristic matrix, identify the structural modal parameters, and obtain the high spatial resolution mode shapes of the structure;

[0009] Step 3: Based on the Gaussian multiscale signal analysis theory, the high spatial resolution mode shapes of the obtained structure are characterized at multiple scales, and the normalized mode curvature energy at each scale is calculated using the second-order difference operator and the Teager energy operator.

[0010] Step 4: Take all measurement points as cluster samples, and normalized mode curvature energy at all scales as multi-dimensional features of the samples. Perform K-Means clustering analysis under multi-scale features. Outliers in the clustering results are identified as damage locations.

[0011] Compared with existing damage identification methods, the advantages of this invention are as follows:

[0012] (1) The present invention uses the phase optical flow algorithm to track the motion of pixels in the structural vibration video. A large number of pixels occupied by the structure in the video image can be obtained by optical flow calculation to obtain their vibration response. Therefore, it has high measurement spatial resolution.

[0013] (2) The present invention maps the high spatial resolution mode shape to a Gaussian multi-scale space, effectively filtering out the interference noise components in the measurement results, and constructs a normalized mode shape curvature energy index to extract and amplify the damage characteristics at each scale, with high identification sensitivity and no dependence on mode shape baseline data.

[0014] (3) The present invention uses the K-Means algorithm to perform multi-dimensional clustering analysis on the damage characteristics of the measurement points at different scales, which reduces the location uncertainty caused by noise interference, and can simultaneously identify multiple structural damage states such as the damaged area and the adjacent damaged area as needed.

[0015] (4) The monitoring equipment of the present invention only requires one industrial camera, which is low in cost and avoids the installation and maintenance of a large number of contact sensors. It has high measurement efficiency and good economic benefits. Attached Figure Description

[0016] Figure 1 This is a flowchart of a structural damage intelligent recognition method based on visual modality multi-scale feature clustering.

[0017] Figure 2a and Figure 2bThese are schematic diagrams of the Gabor wavelet convolution core used in this invention at directional angles of 0 degrees and 90 degrees, respectively.

[0018] Figure 2c and Figure 2d These are schematic diagrams showing the imaginary part of the Gabor wavelet convolution kernel used in this invention at directional angles of 0 degrees and 90 degrees, respectively.

[0019] Figure 3 This is a schematic diagram of the mode shape Gaussian multi-scale spatial mapping used in this invention.

[0020] Figure 4 This is a schematic diagram illustrating the extraction of mode curvature damage characteristics according to the present invention.

[0021] Figure 5a and Figure 5b The figures shown are the results of the present invention in identifying and locating damage in two states in a perforated beam.

[0022] Figure 6a and Figure 6b The figures shown are the results of the present invention in identifying and locating damage in three states in a perforated beam. Detailed Implementation

[0023] The present invention will now be further described with reference to the accompanying drawings.

[0024] like Figure 1 As shown, a method for intelligent structural damage recognition based on visual modality multi-scale feature clustering includes the following steps:

[0025] Step 1: Place a high-speed camera near the target mechanical structure to completely cover the monitoring area with the imaging range, and acquire digital video of the mechanical structure under free vibration. Use a phase optical flow algorithm to track the structural motion frame by frame, perform high spatial resolution visual measurement of the structure's vibration response, and assemble the data to obtain the structure's vibration information matrix.

[0026] Each frame of the image is convolved using a two-dimensional Gabor wavelet kernel, decomposing the vibration video frame by frame into local phase and amplitude spectra. The general expression for constructing a two-dimensional Gabor wavelet is:

[0027]

[0028]

[0029] In the formula, u k v kLet be the independent coordinate variables of the two directions of the convolution kernel, λ be the sine wavelength of the wavelet, θ be the direction angle of the wavelet, ψ be the phase shift of the sine wave, σ be the standard deviation of the Gaussian function used to adjust the sine wave, and γ be the spatial aspect ratio that determines the ellipticity of the wavelet convolution kernel. For the real part of the two-dimensional Gabor wavelet convolution kernel, The imaginary part of the two-dimensional Gabor wavelet convolution kernel, where i is the imaginary unit. These are two coordinate variables on the wavelet direction angle.

[0030] The process of converting each frame of an image into a local phase spectrum and amplitude spectrum through convolution can be represented as:

[0031]

[0032] In the formula, A θ (x,y,t) represents the local amplitude at position (x,y) at time t, and φ θ (x,y,t) represents the local phase at position (x,y) at time t, and I(x,y,t) represents the pixel intensity at position (x,y) at time t. x and y are the pixel coordinates of the measurement point in the image, respectively.

[0033] For each measurement point, the local phase remains unchanged during the motion, meaning the isomorphic contour motion information of the local phase corresponds to the displacement information of the measurement point:

[0034] φ θ (x, y, t) = const

[0035] Taking the partial derivative of the local phase with respect to time, we have the following equation:

[0036]

[0037] In the formula, u and v are the horizontal and vertical pixel coordinate offsets of the measurement point between two frames, respectively, which are the optical flow.

[0038] When θ = 0 and θ = 90°, the Gabor wavelet convolution kernel is as follows: Figure 2a , Figure 2b , Figure 2c and Figure 2d As shown, approximately as well as Therefore, the optical flow at the measurement point can be solved as follows:

[0039]

[0040]

[0041] The optical flow between each frame and the first frame is calculated to obtain the vibration displacement signal. The vibration displacement signals of all measurement points are then assembled to obtain the structural vibration information matrix [δ]. N×T ], where N is the number of measurement points and T is the number of video frames.

[0042] Step 2: Using the Ibrahim time-domain method, the vibration information matrix is ​​sampled three times with a time delay to construct a mathematical model of the feature matrix, identify the modal parameters, and obtain the high spatial resolution vibration modes of the structure.

[0043] According to the modal superposition principle of vibration theory, the structural vibration information matrix under free decaying vibration state can be represented as the superposition of mode shapes of each order, that is:

[0044] [δ N×T ]=[Φ N×r ][q r×T ]

[0045] In the formula, [δ N×T ] represents the structural vibration information matrix, [Φ N×r ] is the modal shape matrix, [q r×T ] is the modal response matrix, N is the number of measurement points, r is the modal order, and T is the number of video frames.

[0046] The vibration information matrix was sampled three times with a time delay. During the first sampling, the vibration response data from the first S frames of all measurement points were collected to obtain the first sampling data matrix.

[0047]

[0048] In the formula, S is the number of sampling frames.

[0049] During the second and third sampling, based on the first sampling, delayed sampling with delays of Δt and 2Δt is performed respectively to obtain the second sampling matrix. and the third sampling matrix

[0050]

[0051]

[0052] In the formula, Δt and 2Δt are the number of delayed frames for the second and third sampling, respectively.

[0053] The three data matrices obtained from the three samplings are combined pairwise to obtain the combined matrix. and And construct the characteristic matrix [D] 2N×2N ]:

[0054]

[0055]

[0056]

[0057] By solving the characteristic matrix [D] 2N×2N The eigenvalues ​​and eigenvectors of the structure are given by the vector φ, which is composed of the first N elements of the eigenvectors. This vector is the high spatial resolution mode shape of the structure.

[0058] Step 3: Based on the multi-scale signal analysis theory, the obtained high spatial resolution mode shape is mapped to the Gaussian multi-scale space, and at each scale, the normalized mode shape curvature energy is calculated using the second-order difference operator and the Teager energy operator.

[0059] By convolving the high spatial resolution mode shapes of the structure with a set of one-dimensional Gaussian functions with variable scaling parameters, the high spatial resolution mode shapes are mapped to a multi-scale space, such as... Figure 3 As shown. This process can be represented as:

[0060]

[0061]

[0062] In the formula, Y i,ο (ζ) represents the i-th mode shape at position ζ in the σ scale, where ζ is the one-dimensional coordinate of the mode shape vector. The kernel function is a one-dimensional Gaussian convolution kernel, φ. i (ζ) represents the i-th mode shape.

[0063] Local damage to a structure can lead to a decrease in local stiffness, and since mode shape curvature is closely related to structural stiffness, the mode shape curvature at the point of damage can undergo a local abrupt change, such as... Figure 4 As shown. Damage features are extracted using modal curvature. Modal curvature is calculated from the second derivative of the modal shape. Considering that the obtained high spatial resolution modal shape is a discrete signal, the second-order central difference operator is used to perform second-order differentiation on the discrete signal. At each scale, the modal curvature is calculated as follows:

[0064]

[0065] In the formula, C i,σ (ζ) represents the mode curvature of the i-th mode shape under the σ-scale characterization, and l is the calculation step size.

[0066] To enhance the local singularity of the damage characteristics, the Teager energy operator is used to calculate the modal curvature energy at each scale:

[0067] Ei,σ (ζ)=(C i,σ (ζ)) 2 -C i,σ (ζ+l)C i,σ (ζ-l)

[0068] In the formula, E i,σ (ζ) represents the mode curvature energy of the i-th mode shape under the σ-scale characterization.

[0069] To facilitate multi-scale feature clustering, the modal curvature energy is normalized at each scale, compressing its value range to [0,1] to obtain the normalized modal curvature energy:

[0070]

[0071] In the formula, Let be the normalized mode curvature energy of the i-th mode shape under the σ-scale characterization.

[0072] Step 4: Take all measurement points as samples, and normalized mode curvature energy at all scales as multidimensional features of the samples. Perform K-Means clustering analysis on multi-scale features. Outliers in the clustering results are identified as damage locations.

[0073] If all measurement points are taken as cluster samples, and the normalized mode curvature energy of each measurement point at all scales is taken as the multidimensional feature coordinates of the sample, then all cluster samples can be represented as:

[0074]

[0075] In the formula, is the cluster sample of the ζ-th measurement point of the i-th mode shape, and s is the total number of Gaussian scales.

[0076] Cluster analysis is performed using all cluster samples as input, and the steps are as follows:

[0077] (1) First, set the number of clusters of the samples to K, and randomly select the multidimensional feature coordinates of K samples as the cluster centers of each category:

[0078]

[0079] In the formula, r is the k-th cluster center of the i-th mode shape. k Let k be the position of the kth random sample.

[0080] (2) For each cluster sample, calculate its Euclidean distance to each cluster center. And the clustered samples are assigned to the category of the nearest cluster center:

[0081]

[0082] (3) For each cluster category, calculate the mean of the feature coordinates of all cluster samples and use this mean as the new cluster center coordinates for that category.

[0083]

[0084] In the formula, Let k be the k-th cluster center after the i-th mode shape is updated. th cluster represents the k-th category.

[0085] (4) Repeat steps (2) and (3) until the cluster center no longer changes or the maximum number of iterations is reached; outliers in the clustering results are identified as damage locations.

[0086] Based on structural health monitoring requirements, if the structure only needs to be divided into two states, [healthy, damaged], then K=2 is used, and outliers in the clustering results are identified as damaged locations. However, if it needs to be further divided into three states, [healthy, near-damaged, damaged], then K=3 is used, and outliers in the clustering results are identified as damaged locations and near-damaged locations.

[0087] Using a typical mechanical structure—a steel beam—as the experimental object, with a length of 500 mm, a width of 20 mm, and a thickness of 5 mm, a 5 mm diameter through hole was introduced at a position of 0.4 of its relative length. One end of the beam was fixedly clamped onto a vibration test bench. Under free vibration, the structural damage intelligent recognition method based on visual modal multi-scale feature clustering of this invention was used to identify damage under two and three states. The results are as follows. Figure 5a and Figure 5b , Figure 6a and Figure 6b As shown in the figure, the results demonstrate that this invention can not only identify and locate damage to mechanical structures in actual noise interference environments, but also further identify areas adjacent to the damage.

Claims

1. A method for intelligent structural damage recognition based on visual modality multi-scale feature clustering, characterized in that, Includes the following steps: Step 1: Acquire high frame rate digital video of the structure under free vibration state, select the pixels of the imaging area corresponding to the structure as measurement points, and use the phase optical flow algorithm for motion tracking, calculate the vibration response of the structure and assemble the measurement points to obtain the structural vibration information matrix. Step 2: Using the Ibrahim time-domain method, the vibration information matrix is ​​sampled three times with a time delay. A mathematical model of the characteristic matrix is ​​constructed to identify modal parameters and obtain high spatial resolution mode shapes of the structure. Step 3: Based on the Gaussian multiscale signal analysis theory, the high spatial resolution mode shapes of the obtained structure are characterized at multiple scales, and the normalized mode curvature energy at each scale is calculated using the second-order central difference operator and the Teager energy operator. Step 4: Take all measurement points as cluster samples, and normalized mode curvature energy at all scales as multidimensional features of the samples. Perform K-Means clustering analysis under multi-scale features. Outliers in the clustering results are identified as damage locations.

2. The intelligent structural damage recognition method based on visual modality multi-scale feature clustering according to claim 1, characterized in that, In step 1, a phase optical flow algorithm is used for motion tracking, the vibration response of the structure is calculated, and measurement points are assembled to obtain the structural vibration information matrix; specifically as follows: Each frame of the image is convolved using a two-dimensional Gabor wavelet convolution kernel, decomposing the video frame by frame into local phase and amplitude spectra. The expression for the two-dimensional Gabor wavelet is as follows: In the formula, These are the independent coordinate variables for the two directions of the convolution kernel. The sine wavelength of the wavelet is given. The direction angle of the wavelet. This represents the phase shift of a sine wave. The standard deviation of the Gaussian function used to adjust the sine wave. To determine the spatial aspect ratio of the wavelet convolution kernel ellipticity, For the real part of the two-dimensional Gabor wavelet convolution kernel, The imaginary part of the two-dimensional Gabor wavelet convolution kernel. The imaginary unit, These are two coordinate variables on the wavelet direction angle; The process of converting each frame of an image into a local phase spectrum and amplitude spectrum through convolution can be represented as follows: In the formula, for t Time is located Local amplitude of position, for t Time is located Local phase of position, for t Time is located Pixel intensity at location, x , y These are the pixel coordinates of the measurement points within the image, representing the horizontal and vertical coordinates respectively. For each measurement point, the local phase remains unchanged during the motion, meaning the isomorphic contour motion information of the local phase corresponds to the displacement information of the measurement point: Taking the partial derivative of the local phase with respect to time, we have the following equation: In the formula, These are the horizontal and vertical pixel coordinate offsets of the measurement point between two frames, i.e., optical flow; when as well as At that time, approximately as well as Therefore, the optical flow at the measurement point is calculated as follows: The optical flow between each frame and the first frame is calculated to obtain the vibration displacement signal. The vibration displacement signals of all measurement points are then assembled to obtain the structural vibration information matrix. ,in, To measure the number of points, T This represents the number of video frames.

3. The intelligent structural damage recognition method based on visual modality multi-scale feature clustering according to claim 1, characterized in that, In step 2, According to the modal superposition principle of vibration theory, the structural vibration information matrix under free decaying vibration state is represented as the superposition of mode shapes of each order, that is: In the formula, For structural vibration information matrix, The modal shape matrix, Here is the modal response matrix. N To measure the number of points, The modal order is... T This refers to the number of video frames. The vibration information matrix is ​​sampled three times with a time delay. In the first sampling, the vibration response data of the first S frames of all measurement points are taken to obtain the first sampling data matrix. : In the formula, This represents the number of sampling frames. For the second and third sampling, a delay is applied based on the first sampling. and Delayed sampling is used to obtain the second sampling matrix. and the third sampling matrix : In the formula, and These are the number of delayed frames for the second and third sampling, respectively; The three data matrices obtained from the three samplings are combined pairwise to obtain the combined matrix. and And construct the characteristic matrix : By solving the characteristic matrix The eigenvalues ​​and eigenvectors, the first eigenvectors A vector consisting of elements This refers to the high spatial resolution mode shape of the structure.

4. The intelligent structural damage recognition method based on visual modality multi-scale feature clustering according to claim 1, characterized in that, Step 3 is described in detail below: The high spatial resolution mode shapes of the structure are convolved with a set of one-dimensional Gaussian functions with variable scaling parameters, mapping them to a multi-scale space: In the formula, for Position at No. i First-order mode shape in Characterization at scale The one-dimensional coordinates of the mode shape vector. It is a one-dimensional Gaussian convolution kernel function. For the first i First-order mode shape; Local structural damage leads to a decrease in local stiffness, and modal curvature is closely related to structural stiffness. Modal curvature is used to detect damage characteristics. Modal curvature is calculated from the second derivative of the modal shape. Considering that the obtained high spatial resolution modal shapes are discrete signals, a second-order central difference operator is used to perform second-order differentiation. At each scale, the modal curvature is calculated using the following formula: In the formula, for Position at No. Mode shape in Modal curvature under scale characterization To calculate the step size; To enhance the local singularity of the damage characteristics, the Teager energy operator is used to calculate the modal curvature energy at each scale: In the formula, for Position at No. Mode shape in Modal curvature energy under scale characterization; To facilitate multi-scale feature clustering, the modal curvature energy is normalized at each scale, compressing its value range to [value range missing]. Obtain the normalized mode curvature energy: In the formula, for Position at No. Mode shape in Normalized mode curvature energy under scale characterization.

5. The intelligent structural damage recognition method based on visual modality multi-scale feature clustering according to claim 1, characterized in that, Step 4 is described in detail below: If all measurement points are taken as cluster samples, and the normalized mode curvature energy of each measurement point at all scales is taken as the multidimensional feature coordinates of that sample, then all cluster samples are represented as follows: In the formula, For the first The first mode of vibration Clustering samples of measurement points, for Position at No. Mode shape in Normalized mode curvature energy under scale characterization The number of Gaussian scales; Cluster analysis is performed using all cluster samples as input, and the steps are as follows: (1) First, set the number of clusters for the cluster analysis to [number]. and randomly select The multidimensional feature coordinates of each cluster sample serve as the cluster center for each category: In the formula, For the first The first mode of vibration Cluster centers, For the first A random sample location; (2) For each cluster sample, calculate its Euclidean distance to each cluster center. And the clustered samples are assigned to the category of the nearest cluster center: (3) For each cluster category, calculate the mean of the feature coordinates of all cluster samples, and use this mean as the new center coordinate of that cluster category. In the formula, For the first The updated first mode shape Cluster centers, Indicates the first One category; (4) Repeat steps (2) and (3) until the cluster center no longer changes or the maximum number of iterations is reached; the outliers in the clustering results are determined to be the damage locations.

6. The intelligent structural damage recognition method based on visual modality multi-scale feature clustering according to claim 5, characterized in that, Based on the structural health monitoring requirements, if the structure only needs to be divided into two states: [healthy] and [damaged], then take... Outliers in the clustering results are identified as damage locations; if further classification into three states—[healthy, near-damage, damage]—is needed, then... Outliers in the clustering results are identified as the location of the damage and its vicinity.

Images