Signal reconstruction method and device for vision-based structural vibration mode recognition under environmental excitation

Abstract

The present invention discloses a signal reconstruction method and device for vision-based structural vibration mode identification under environmental excitation. The method obtains a source image of the structure to be tested, extracts the displacement signal of the region of interest of the structure to be tested based on the regions of interest of several frames of the source image using a template matching method combined with sub-pixel technology, reconstructs the displacement signal to obtain a reconstructed displacement signal, and obtains a reconstructed acceleration signal based on the reconstructed displacement signal. Based on the acceleration signal, the vibration mode of the region of interest is obtained using a natural excitation technique combined with an intrinsic system implementation algorithm. The above steps are repeated to obtain the vibration mode of all regions of interest in the source image of the structure to be tested, and then splices them to obtain the vibration mode of the structure to be tested. This method ensures the noise resistance and robustness of low-level vibration signals. The reconstructed acceleration signal provides significant advantages for high-frequency modal identification based on noisy high-speed camera data, and has broad application prospects in full-field SHM.

Description

Technical Field

[0001] The present invention relates to the field of engineering structure health monitoring, and in particular to a signal reconstruction method and device for visual-based structural vibration mode recognition under environmental excitation. Background Art

[0002] Structural health monitoring (SHM) aims to provide important information for structural assessment and ensure the safe operation of engineering structures by monitoring various physical parameters, evaluating the condition and performance of the structure, and guiding routine inspection and maintenance. Over the past three decades, researchers from around the world have invested considerable time and effort in developing novel sensing technologies for SHM research. With significant advances in optics and computer science, vision-based sensing and monitoring technologies have gained popularity in the SHM field due to their advantages as non-contact, long-distance, and multi-point measurement.

[0003] Photogrammetry is a measurement method that uses photographs or digital images to extract the geometry, displacement, and deformation of a structure. In the 1980s, Peters and Ranson pioneered the use of photogrammetry to detect stresses and strains during structural deformation. Although research on vision-based SHM is still in its early stages, significant efforts have been made to identify quantitative indicators of structural condition from vision-based data. To date, photogrammetry has been applied in a variety of fields, including structural displacement measurement, strain / stress monitoring, vibration response monitoring, modal property identification, model updating, and damage detection. Dynamic response measurement and identification of structural modal properties (natural frequencies, damping ratios, and mode shapes) are widely used in SHM. In contrast to accelerometers and vibrometers, cameras can provide spatially dense information for modal analysis. By examining high-speed video sequences, Mas et al. developed a technique for simultaneously detecting vibration frequencies at multiple points. Yoon et al. studied the vibration of a laboratory frame structure and compared modal parameters obtained through visual identification with those obtained using accelerometers.

[0004] It is worth noting that three types of photogrammetry techniques have been developed: template matching methods, optical flow methods, and phase-based methods. Template matching methods are used to track the motion of targets in video image sequences, often in combination with sub-pixel techniques. The method has two main components: a template image T and a source image I. Optical flow theory explains how image intensity and structural motion are related, and optical flow methods can be used to obtain the full-field displacement of a structure from the image intensity data. This method is applicable to situations where there are no artificially mounted targets and can obtain the full-field mode shapes of the structure. Phase-based methods are used to quickly recover displacements from the phase space of the image and are generally more resilient to noise and interference than optical flow-based systems. In summary, all three methods are based on the following basic premise: first, the displacements are extracted from the visual data; then, the modal parameters are extracted from the displacement signal using existing techniques.

[0005] When performing modal analysis of long-term SHM, the monitored structure is typically subject only to environmental excitation, resulting in minimal vibrations and the extracted dynamic response being susceptible to measurement noise. In fact, the displacement amplitude of a structure subjected to environmental excitation can be as small as the noise level of a vision-based measurement system. Therefore, while the structure's natural frequency can be approximately estimated using Fourier transforms, quantifying the mode shapes using vision-based techniques is difficult because the extracted dynamic responses are often insufficiently accurate. Furthermore, reconstructing the structure's mode shapes is only possible if the dynamic responses at multiple points are accurately acquired. Unfortunately, vibrations at points near nodes or fixed boundaries are very small and are masked by measurement noise, rendering the mode shape values at these points unusable. Therefore, a key issue in vision-based modal shape identification of structures subjected to environmental excitation is improving the noise immunity and robustness of low-level vibration signals.

[0006] A straightforward idea is to adjust the focal length of the camera so that the limited field of view (FOV) is focused on a local area of the structure, so that small amplitude vibrations can be captured more accurately. However, this means that a single camera cannot monitor the complete field of view of the dynamic response of the entire structure, and therefore can only obtain the modal vibration shape values at the points in the FOV. To address this limitation, research has been conducted to develop multi-view systems for static and dynamic displacement assessment. The structure of interest is usually divided into many parts, each covering a different area of the structure, and the vibration shape of the entire structure can then be assembled accordingly. For local areas near nodes or fixed boundaries, it is acknowledged that the motion accuracy of some points in the FOV can be guaranteed using this approach; however, there are still some points in the same view whose motion cannot be accurately extracted, resulting in non-negligible errors in the vibration shape assembly of the entire structure.

[0007] Motion amplification methods have recently been proposed to further improve the signal-to-noise ratio (SNR) in low-amplitude vibrations. Chen et al. combined phase-based motion estimation (PME) with motion enhancement techniques to determine the frequencies and operating deflection modes of laboratory-scale benchmark structures, including cantilever beams and pipes. Yang et al. developed an unsupervised learning method for modal analysis by combining blind source separation with PME and motion enhancement. However, motion amplification techniques require preprocessing of the source video, which requires a large amount of storage space. Furthermore, due to technical limitations, artifacts appear in the amplified video. Subsequently, a hybrid identification method combining a high-speed camera with an accelerometer was proposed, and a dynamic substructure method was used to improve the estimation accuracy of the mode shape. However, this method loses its generality if the structure is not suitable for mounting accelerometers. Summary of the Invention

[0008] In response to the above-mentioned technical problems, the embodiment of the present application aims to propose a signal reconstruction method and device for visual-based structural vibration mode recognition under environmental excitation, so as to solve the technical problems mentioned in the above background technology section.

[0009] In a first aspect, the present invention provides a signal reconstruction method for visually identifying structural vibration modes under environmental excitation, comprising the following steps:

[0010] S1, obtaining a source image of the structure to be measured, and extracting the displacement signal of the region of interest of the structure to be measured by a template matching method combined with sub-pixel technology based on the region of interest of several frames of source images;

[0011] S2, reconstructing the displacement signal to obtain a reconstructed displacement signal, and obtaining a reconstructed acceleration signal according to the reconstructed displacement signal;

[0012] S3, according to the acceleration signal, the natural excitation technology combined with the eigensystem implementation algorithm is used to obtain the vibration mode of the region of interest;

[0013] S4, repeating steps S1-S3 to obtain the mode shapes of all regions of interest in the source image of the structure to be measured, and splicing them to obtain the mode shapes of the structure to be measured.

[0014] Preferably, before step S1, the method includes:

[0015] Before measurement, camera calibration is performed to determine the relationship between pixel coordinates and physical coordinates. When the image plane is parallel to the object surface, the scale factor method is used for camera calibration. The scale factor on the x-axis can be expressed as follows:

[0016]

[0017] Where d is the pixel length of the structure in the image, D is the real length of the structure, z is the distance between the camera and the structure, and f is the focal length of the camera.

[0018] Preferably, the source image includes several regions of interest, and each region of interest includes a template image.

[0019] Preferably, step S1 specifically includes:

[0020] The region of interest containing the template image is determined in the source image. The template image is moved pixel by pixel to determine the matching area. The template image is compared with the region of interest. The frequency domain cross-correlation between the template image and the region of interest is calculated to determine the position of the template image in the source image. The displacement signal is obtained based on the pixel difference between the template image and two adjacent frames of the source image.

[0021] Preferably, calculating the frequency domain cross-correlation between the template image and the region of interest, determining the position of the template image in the source image, and obtaining a displacement signal based on the pixel difference between the template image and two adjacent source image frames specifically includes:

[0022] Consider a template image I(x,y) and a region of interest T(x,y) of the same dimension M×N. Apply Fourier transform to I(x,y) and T(x,y). The cross-correlation between the two is expressed as:

[0023]

[0024] Wherein, Δx and Δy are the coordinate offsets in the x and y directions, P(u,v) is the discrete Fourier transform (DFT) of the template image, P(u,v) is the discrete Fourier transform of the template image, Q(u,v) is the discrete Fourier transform of the source image, * represents the complex conjugate; u and v represent the abscissa and ordinate of the image in the frequency domain after the Fourier transform, respectively; x' represents the abscissa of the image in the spatial domain after the offset Δx; y' represents the ordinate of the image in the spatial domain after the offset Δy; M is the length of the template image, and N is the width of the template image;

[0025] By searching the cross-correlation F corr The pixel-level coordinates of the template image in the region of interest (x p ,y p ), calculate the pixel-level coordinates (x p ,y p ) by searching the cross-correlation F corr The maximum value of the sub-pixel coordinates of the template image in the region of interest (x sp ,y sp );

[0026] The first frame of the source image is used as the reference frame, and the sub-pixel coordinates of the reference frame are (x sp,1 ,y sp,1 ), the sub-pixel coordinates of the template image in the region of interest of each new frame source image are (x sp,i ,y sp,i ), the sub-pixel displacement signal of the region of interest of the structure to be measured is expressed as: y(t)=(x sp,i ,y sp,i )-(x sp,1 ,y sp,1 ).

[0027] Preferably, step S2 specifically includes:

[0028] Consider the sub-pixel displacement signal y(t) extracted at a certain point in the region of interest of the structure to be measured, and its Fourier transform is expressed as:

[0029]

[0030] Among them, F y (ω) is the spectrum of the displacement signal. Based on the above formula, a frequency domain operator is proposed and defined as follows:

[0031]

[0032] Among them, β represents the proportional factor, ω m =2πf m is the mth angular frequency of the structure, Δω is the half bandwidth;

[0033] The time history of the displacements is then reconstructed by using an inverse Fourier transform to generate the following equation:

[0034]

[0035] in, The reconstructed displacement signal is obtained by applying the central difference to the reconstructed displacement, and the reconstructed acceleration signal is expressed as:

[0036]

[0037] Among them, the reconstructed acceleration signal is Δt is the corresponding time interval in the time domain. When the Fourier transform is applied to When , we get:

[0038]

[0039] Preferably, step S3 specifically includes:

[0040] In the discrete time domain, the dynamics of a linear and time-invariant dynamical system with n degrees of freedom is described by the state-space equations as follows:

[0041] p(k+1)=A d p(k)+B d u(k);

[0042] q(k)=C d p(k)+D d u(k);

[0043] Among them, k represents the kth discrete time step, A d ∈R 2n×2n 、B d ∈R 2n×l 、C d ∈R m×2n and D d ∈R m×l They are the state matrix, input matrix, output matrix and transfer matrix, respectively, where the state matrix, input matrix and transfer matrix are known quantities related to the mass stiffness and damping of the structure; p(k)∈R 2n×l ,q(k)∈R m×l and u(k)∈R l×1 They are 2n-dimensional input vector, m-dimensional output vector, and l-dimensional excitation vector respectively; l represents the number of inputs, m represents the number of outputs, and 2n represents the order of the state matrix; the environmental excitation is used as the input vector p(k), and the reconstructed acceleration signal is used as the output vector q(k), and the output matrix C is obtained. d ;

[0044] When the output vector q(k) is specified as the impulse response, the following Hankel matrix is formed:

[0045]

[0046] Where α = 1, 2, 3…, β = 1, 2, 3…, each element in the Hankel matrix is an m×n dimensional matrix, so the order of the Hankel matrix is αm×βn;

[0047] Obtain similarity matrix by two Hankel matrices with the same time step The calculation is as follows:

[0048]

[0049] Among them, V k-1 、U k-1 and Γ k-1 Satisfies the singular value decomposition of the Hankel matrix, that is:

[0050]

[0051] Similarity Matrix The eigenvalue decomposition of can be used to calculate the natural frequencies as follows:

[0052]

[0053] Among them, Λ d is a diagonal matrix, Λ d yes The eigenvalue matrix, Ψ is the eigenvector matrix, Denote the nth order angular frequency and its complex conjugate respectively, and the mth natural frequency f of the structure to be tested is obtained using the following equation m :

[0054]

[0055] The sampling frequency is f s , the eigenvalue matrix is obtained through the natural frequency, and the eigenvector matrix Ψ is calculated based on the eigenvalue matrix;

[0056] The mode shapes for the region of interest are generated using the following equations:

[0057] Φ=C d Ψ.

[0058] In a second aspect, the present invention provides a signal reconstruction device for visually identifying structural vibration modes under environmental excitation, comprising:

[0059] a displacement signal extraction module configured to obtain a source image of the structure to be measured, and extract a displacement signal of the region of interest of the structure to be measured by a template matching method combined with sub-pixel technology based on the region of interest of a plurality of frames of the source image;

[0060] a signal reconstruction module configured to reconstruct the displacement signal to obtain a reconstructed displacement signal, and obtain a reconstructed acceleration signal based on the reconstructed displacement signal;

[0061] A local mode shape determination module is configured to obtain the mode shape of the region of interest using a natural excitation technique combined with an eigensystem implementation algorithm based on the acceleration signal;

[0062] The splicing module is configured to repeatedly execute the displacement signal extraction module to the local vibration shape determination module to obtain the vibration shapes of all the regions of interest in the source image of the structure to be measured, and splice them to obtain the vibration shapes of the structure to be measured.

[0063] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0064] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] (1) The signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation proposed in the present invention can realize dynamic response extraction and modal vibration mode recognition under environmental excitation. Without amplifying the original video, the signal is directly processed, making the modal recognition more efficient. It has the advantages of short operation time and moderate storage space requirements.

[0067] (2) The proposed signal reconstruction method for vision-based structural vibration mode recognition under environmental excitation ensures noise immunity and robustness of low-level vibration signals. The reconstructed acceleration signal provides significant advantages for high-frequency modal recognition based on noisy high-speed camera data. This signal reconstruction method has the potential to be developed into a modal mode recognition method under environmental excitation and therefore has broad application prospects in full-field SHM.

[0068] (3) The signal reconstruction method for vision-based structural vibration mode identification under environmental excitation proposed in the present invention can identify the first two natural frequencies and modal vibration modes even when the vibration amplitude is as small as 0.01 mm, while the traditional vision-based method can only identify modal parameters with vibration amplitude greater than 0.06 mm. It also performs well in experimental tests considering four camera settings including different focal lengths and object distances, indicating that this method has great potential for application under various conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0070] Figure 1 is a diagram of an exemplary device architecture to which an embodiment of the present application may be applied;

[0071] Figure 2Schematic diagram of the flow of a signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation according to an embodiment of the present application;

[0072] Figure 3 Figure 1 is a diagram of the experimental setup for the signal reconstruction method for vision-based structural vibration mode identification under environmental excitation according to an embodiment of the present application, wherein (a) is a view of the overall experimental scene; (b) is an enlarged side view of the camera and tripod; (c) is a detailed view of the cantilever steel beam; and (d) is a detailed view of the target and accelerometer.

[0073] Figure 4 This is a time history diagram of the accelerometer measurement of the signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation in an embodiment of the present application; wherein (a) is the time domain diagram of the acceleration signal at point 3; (b) is the frequency domain diagram of point 3;

[0074] Figure 5 The vibration modes of acceleration data of the signal reconstruction method of the visual-based structural vibration mode recognition under environmental excitation in the embodiment of the present application; wherein (a) is the first-order vibration mode; (b) is the second-order vibration mode;

[0075] Figure 6 A schematic diagram of a region of interest in a signal reconstruction method for vision-based structural vibration mode recognition under environmental excitation according to an embodiment of the present application;

[0076] Figure 7 The vibration signal of point 3 obtained by the camera-based measurement method of the signal reconstruction method of the visual-based structural vibration mode recognition under environmental excitation in an embodiment of the present application; wherein (a) is the displacement signal; (b) is the corresponding power spectral density;

[0077] Figure 8 The vibration signal of point 2 obtained by the camera-based measurement method of the signal reconstruction method of the visual-based structural vibration mode recognition under environmental excitation of the embodiment of the present application: (a) is the time history of the displacement of point 2 for 25 seconds; (b) is the time history of the displacement of point 2 for the first 4 seconds;

[0078] Figure 9 This is a frequency domain diagram of the displacement signal of point 2 in the signal reconstruction method of the visual-based structural vibration mode recognition under environmental excitation according to an embodiment of the present application;

[0079] Figure 10 The signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation of an embodiment of the present application shows the displacement of point 2 and the corresponding spectra of different excitation amplitudes;

[0080] Figure 11This is the time history of the acceleration directly calculated and reconstructed at point 2 in the signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation in an embodiment of the present application; wherein (a) is the directly calculated acceleration; (b) is the reconstructed acceleration;

[0081] Figure 12 This is a comparison result diagram of the direct derivation and reconstructed acceleration spectrum of the signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation in an embodiment of the present application;

[0082] Figure 13 The comparison results of the modal vibration shapes extracted from different dynamic responses of the signal reconstruction method of the visual-based structural vibration shape recognition under environmental excitation of the embodiment of the present application and different excitation output voltages;

[0083] Figure 14 Schematic diagram of ROI of point 2 in the video with various camera parameter settings for the signal reconstruction method of vision-based structural vibration mode recognition under environmental excitation according to an embodiment of the present application;

[0084] Figure 15 Comparison of modal vibration shapes extracted from different dynamic responses under different camera parameters for the signal reconstruction method of the vision-based structural vibration shape recognition under environmental excitation according to an embodiment of the present application;

[0085] Figure 16 Schematic diagram of a signal reconstruction device for visual-based structural vibration mode recognition under environmental excitation according to an embodiment of the present application;

[0086] Figure 17 It is a structural diagram of a computer device suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0087] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0088] Figure 1 An exemplary device architecture 100 is shown to which the signal reconstruction method for vision-based structural vibration mode recognition under environmental excitation or the signal reconstruction device for vision-based structural vibration mode recognition under environmental excitation according to the embodiments of the present application can be applied.

[0089] like Figure 1As shown, the device architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0090] Users can use terminal devices 101, 102, 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications, such as data processing applications and file processing applications, can be installed on terminal devices 101, 102, 103.

[0091] Terminal devices 101, 102, and 103 can be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules (for example, software or software modules used to provide distributed services), or they can be implemented as a single software or software module. No specific limitations are given here.

[0092] The server 105 may be a server that provides various services, such as a background data processing server that processes files or data uploaded by the terminal devices 101, 102, and 103. The background data processing server may process the acquired files or data and generate processing results.

[0093] It should be noted that the signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation provided in the embodiment of the present application can be executed by the server 105, or by the terminal devices 101, 102, and 103. Accordingly, the signal reconstruction device for visual-based structural vibration mode recognition under environmental excitation can be set in the server 105, or in the terminal devices 101, 102, and 103.

[0094] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the above description is merely illustrative. Any number of terminal devices, networks, and servers may be provided as needed. If the processed data does not need to be acquired remotely, the above-described apparatus architecture may not include a network, but only require servers or terminal devices.

[0095] Figure 2 A signal reconstruction method for visually identifying structural vibration modes under environmental excitation provided by an embodiment of the present application is shown, comprising the following steps:

[0096] S1, obtaining a source image of the structure to be measured, and extracting a displacement signal of the region of interest of the structure to be measured by a template matching method combined with sub-pixel technology based on the region of interest of several frames of source images.

[0097] Specifically, the template matching method has high stability and noise resistance. When the vibration amplitude of the structure to be measured is very small, higher resolution is usually required. However, the best strategy is to add sub-pixel technology to the template matching method to improve the measurement accuracy. Therefore, the sub-pixel template matching method is a key prerequisite for the extraction of displacement signals during vibration.

[0098] In a specific embodiment, the process before step S1 includes:

[0099] Before measurement, camera calibration is performed to determine the relationship between pixel coordinates and physical coordinates. When the image plane is parallel to the object surface, the scale factor method is used for camera calibration. The scale factor on the x-axis can be expressed as follows:

[0100]

[0101] Where d is the pixel length of the structure in the image, D is the real length of the structure, z is the distance between the camera and the structure, and f is the focal length of the camera.

[0102] In a specific embodiment, the source image contains several regions of interest, each of which contains a template image. Once the camera calibration is completed, the source image can be acquired and then target tracking can be performed. The template image is moved pixel by pixel to determine the matching area, and the template image is compared with the region of interest (ROI) in the source image. The position of the template image in the source image can then be determined, and there are two criteria for estimating the similarity of the template matching method, namely the cross-correlation criterion and the sum of squared differences (SSD) correlation criterion. The former takes into account computational efficiency, stability, and sub-pixel accuracy, so the embodiments of the present application use the cross-correlation criterion for similarity assessment.

[0103] In practical applications, when the structure to be measured is vibrating at low amplitude, a source image with higher resolution is usually preferred. The best strategy is to include sub-pixel technology in the template matching method to improve measurement accuracy. The present invention uses Guizar's sub-pixel technology, which consists of two steps: first, the frequency domain cross-correlation between the template image and the source image is calculated to generate pixel-level template matching points (x p ,y p ); Secondly, the pixel-level template matching point (x p ,y p ) and the sub-pixel coordinates (x sp ,y sp ).

[0104] In a specific embodiment, step S1 specifically includes:

[0105] The region of interest containing the template image is determined in the source image. The template image is moved pixel by pixel to determine the matching area. The template image is compared with the region of interest. The frequency domain cross-correlation between the template image and the region of interest is calculated to determine the position of the template image in the source image. The displacement signal is obtained based on the pixel difference between the template image and two adjacent frames of the source image.

[0106] In a specific embodiment, calculating the frequency domain cross-correlation between the template image and the region of interest, determining the position of the template image in the source image, and obtaining a displacement signal based on the pixel difference between the template image and two adjacent source image frames specifically includes:

[0107] Consider a template image I(x,y) and a region of interest T(x,y) of the same dimension M×N. Apply Fourier transform to I(x,y) and T(x,y). The cross-correlation between the two is expressed as:

[0108]

[0109] Wherein, Δx and Δy are the coordinate offsets in the x and y directions, P(u,v) is the discrete Fourier transform (DFT) of the template image, P(u,v) is the discrete Fourier transform of the template image, Q(u,v) is the discrete Fourier transform of the source image, * represents the complex conjugate; u and v represent the abscissa and ordinate of the image in the frequency domain after the Fourier transform, respectively; x' represents the abscissa of the image in the spatial domain after the offset Δx; y' represents the ordinate of the image in the spatial domain after the offset Δy; M is the length of the template image, and N is the width of the template image;

[0110] By searching the cross-correlation F corr The pixel-level coordinates of the template image in the region of interest (x p ,y p ), calculate the pixel-level coordinates (x p ,y p ) by searching the cross-correlation F corr The maximum value of the sub-pixel coordinates of the template image in the region of interest (x sp ,y sp ); It should be noted that if the peak search is within the (1.5κ, 1.5κ) neighborhood, the sub-pixel resolution can be 1 / κ of the pixel, and a large number of studies have shown that the sub-pixel accuracy ranges from 0.5 to 0.01 pixels.

[0111] The first frame of the source image is used as the reference frame, and the sub-pixel coordinates of the reference frame are (x sp,1 ,y sp,1 ), the sub-pixel coordinates of the template image in the region of interest of each new frame source image are (x sp,i ,y sp,i ), the sub-pixel displacement signal of the region of interest of the structure to be measured is expressed as: y(t)=(x sp,i ,y sp,i )-(x sp,1 ,y sp,1 ).

[0112] S2, reconstructing the displacement signal to obtain a reconstructed displacement signal, and obtaining a reconstructed acceleration signal according to the reconstructed displacement signal.

[0113] The above-mentioned sub-pixel image processing technology can obtain the displacement signal of the region of interest of the structure to be measured. However, it is usually difficult to identify the mode shape from the displacement signals of different points extracted by the visual template matching method. There are generally two reasons for this:

[0114] (1) The resolution of displacements extracted from images is so limited that the displacements are not accurate enough to reconstruct the modal modes using existing modal analysis methods. Subpixel technology has been proposed as a solution to improve displacement resolution, as briefly described above; however, the resolution of the extracted dynamic displacements remains insufficient for low-amplitude vibrations due to environmental excitation. Measurement noise and system noise contaminate the images, further increasing the difficulty of reconstructing the modal modes.

[0115] (2) The displacement amplitude of the high-frequency component is much smaller than that of the low-frequency component, which makes it more difficult to extract high-order vibration modes from the displacement. Therefore, the embodiment of the present application proposes a method to improve the SNR of low-amplitude data by using a signal reconstruction method.

[0116] When a mechanical system with multiple degrees of freedom (MDOF) is subjected to external forces, its equilibrium motion equation can be expressed as:

[0117]

[0118] Where f(t) is the applied excitation force vector, M, C, and K are the mass matrix, damping matrix, and stiffness matrix, respectively. and y(t) are acceleration, velocity and displacement vectors respectively. and speed When applying the Fourier transform, we get:

[0119]

[0120] Where F(·) is the Fourier transform and ω is the angular frequency. As shown above, the amplitude of the acceleration spectrum is magnified by ω compared to the displacement. 2 times. Therefore, acceleration is preferable to displacement when extracting vibration modes, especially for higher-order vibration modes. Traditionally, in modal testing, acceleration can be measured directly by accelerometers; however, for vision-based techniques, acceleration can only be obtained by applying central difference to the extracted displacement. Due to insufficient resolution and inevitable noise, it is almost impossible to calculate sufficiently accurate acceleration from the displacement directly extracted by central difference. Therefore, an embodiment of the present application proposes to reconstruct the displacement signal extracted by the template matching method, and then the reconstructed acceleration can be generated by central difference, which can be further used to identify the first few modal vibration modes.

[0121] In a specific embodiment, step S2 specifically includes:

[0122] Consider the sub-pixel displacement signal y(t) extracted at a certain point in the region of interest of the structure to be measured, and its Fourier transform is expressed as:

[0123]

[0124] Among them, F y (ω) is the spectrum of the displacement signal. Based on the above formula, a frequency domain operator is proposed and defined as follows:

[0125]

[0126] Among them, β represents the proportional factor, ω m =2πf m is the mth angular frequency of the structure, and Δω is the half bandwidth. As can be seen from the above formula, the frequency domain operator keeps the frequency components near the natural frequency and significantly reduces other frequency components, thereby ensuring that the frequency components near the natural frequency dominate.

[0127] The time history of the displacements is then reconstructed by using an inverse Fourier transform to generate the following equation:

[0128]

[0129] in, The reconstructed displacement signal is obtained by applying the central difference to the reconstructed displacement, and the reconstructed acceleration signal is expressed as:

[0130]

[0131] Among them, the reconstructed acceleration signal is Δt is the corresponding time interval in the time domain. When the Fourier transform is applied to When , we get:

[0132]

[0133] As can be seen from the figure, in the reconstructed acceleration signal, the frequency components close to the natural frequency are well preserved, while other frequency components are suppressed. Therefore, using this method can effectively reduce the noise in the original displacement, thereby further accurately identifying the mode vibration shape.

[0134] S3, according to the acceleration signal, the natural excitation technology combined with the eigensystem implementation algorithm is used to obtain the vibration shape of the region of interest.

[0135] Specifically, in order to be applicable to the estimation of modal parameters under small vibrations of structures affected by environmental excitations, an embodiment of the present application proposes a method combining natural excitation technology (NExT) and eigensystem realization algorithm (ERA) to determine the micro-vibration mode of the structure. The essence of ERA is to use the measured free vibration data or the impulse response function of the system to solve a minimum realization of the system, convert the minimum realization into a canonical form in the form of eigenvalues, and finally extract the modal parameters of the system. The displacement signal of the structure under environmental excitation is extracted based on the template matching method combined with sub-pixel technology. When the background noise of the camera masks the vibration response amplitude, the signal reconstruction method is used to convert the original low-amplitude displacement data into a reconstructed acceleration signal, and the NExT-ERA method and mode shape splicing technology are used to identify the mode shape of the entire structure, thereby realizing vision-based structural mode shape recognition under environmental excitation.

[0136] In a specific embodiment, step S3 specifically includes:

[0137] In the discrete time domain, the dynamics of a linear and time-invariant dynamical system with n degrees of freedom is described by the state-space equations as follows:

[0138] p(k+1)=A d p(k)+B d u(k);

[0139] q(k)=C d p(k)+D d u(k);

[0140] Among them, k represents the kth discrete time step, A d ∈R 2n×2n 、B d ∈R 2n×l 、C d ∈R m×2n and D d ∈R m×l They are the state matrix, input matrix, output matrix and transfer matrix, respectively, where the state matrix, input matrix and transfer matrix are known quantities related to the mass stiffness and damping of the structure; p(k)∈R2n×l ,q(k)∈R m×l and u(k)∈R l×1 They are 2n-dimensional input vector, m-dimensional output vector, and l-dimensional excitation vector respectively; l represents the number of inputs, m represents the number of outputs, and 2n represents the order of the state matrix; the environmental excitation is used as the input vector p(k), and the reconstructed acceleration signal is used as the output vector q(k), and the output matrix C is obtained. d ;

[0141] When the output vector q(k) is specified as the impulse response, the following Hankel matrix is formed:

[0142]

[0143] Where α = 1, 2, 3…, β = 1, 2, 3…, each element in the Hankel matrix is an m×n dimensional matrix, so the order of the Hankel matrix is αm×βn;

[0144] Obtain similarity matrix by two Hankel matrices with the same time step The calculation is as follows:

[0145]

[0146] Among them, V k-1 、U k-1 and Γ k-1 Satisfies the singular value decomposition of the Hankel matrix, that is:

[0147]

[0148] Similarity Matrix The eigenvalue decomposition of can be used to calculate the natural frequencies as follows:

[0149]

[0150] Among them, Λ d is a diagonal matrix, Λ d yes The eigenvalue matrix, Ψ is the eigenvector matrix, Denote the nth order angular frequency and its complex conjugate respectively, and the mth natural frequency f of the structure to be tested is obtained using the following equation m :

[0151]

[0152] The sampling frequency is f s , the eigenvalue matrix is obtained through the natural frequency, and the eigenvector matrix Ψ is calculated based on the eigenvalue matrix;

[0153] The mode shapes for the region of interest are generated using the following equations:

[0154] Φ=C d Ψ.

[0155] It is important to note that the impulse response is used to generate the Hankel matrix equations, and if the input vector is an environmental excitation, the response should be converted to an impulse response.

[0156] S4, repeating steps S1-S3 to obtain the mode shapes of all regions of interest in the source image of the structure to be measured, and splicing them to obtain the mode shapes of the structure to be measured.

[0157] Specifically, the vibration mode of each region of interest is obtained through identification through steps S1-S3. When the field of view (FOV) of the camera is limited, the source image of the structure to be measured is usually divided into many parts, each part covering a different area of the structure to be measured. Then, the vibration mode of the entire structure can be assembled accordingly and used as the final recognition result after signal reconstruction.

[0158] This paper proposes a signal reconstruction method for vision-based structural vibration mode recognition under environmental excitation to identify the mode vibration mode of structures affected by low-amplitude environmental excitation. The method generally includes three steps. First, a template matching method is combined with a sub-pixel method to extract the displacement of different points on the structure through a series of source images in the video. Then, a frequency domain operator is designed to keep the frequency component near the natural frequency and significantly reduce other frequency components. It is applied to the extracted dynamic displacement to reconstruct the displacement signal, and the corresponding acceleration signal is obtained through central difference. Finally, the NExT-ERA method and vibration mode splicing technology are used to identify the vibration mode of the entire structure. This method can be used for dynamic response extraction and modal vibration mode recognition of structures under micro-vibration, and has strong robustness to resist noise interference. Preliminary research results show that the signal reconstruction method has the potential to develop into a modal vibration mode recognition method under environmental excitation, and therefore has broad application prospects in full-field SHM.

[0159] To verify the feasibility of the proposed visual-based signal reconstruction method for structural vibration mode identification under environmental excitation, a steel cantilever beam measuring 2 mm × 20 mm × 1000 mm was selected for laboratory testing. Five targets were mounted on the beam for visual measurement, and five accelerometers were installed at the same locations on the beam. This allowed for reference to traditional modal analysis methods that directly use acceleration. A vibrator was installed near the fixed end of the beam, and random excitation was applied to simulate environmental excitation. A camera was mounted on a tripod and used an LED light source to simulate ambient lighting. The resulting vibrations were monitored by the camera and accelerometers. Specifically, video was recorded at 50 frames per second with a resolution of 2448 × 2048. The acceleration was sampled at 50 Hz, and the duration of both the video and the acceleration was 25 seconds. Figure 3 (a) shows the entire experimental setup, Figure 3 (b) to 3(d) show the magnified views of the camera, cantilever steel beam, target, and accelerometer, respectively.

[0160] Two series of tests were designed: For the first series, the output voltage of the random excitation was increased from 300mV to 1500mV, while the focal length was maintained at 35mm and the object distance was 160cm. The purpose was to investigate the performance of the proposed method in reconstructing the mode shapes of structures vibrating with different amplitudes due to environmental excitation. For the second series, the output voltage of the random excitation was maintained at 1500mV, the focal length was varied from 12mm to 35mm, and the object distance was varied from 100cm to 200cm. The purpose was to investigate the feasibility of the proposed method with different camera settings.

[0161] Since traditional modal analysis using directly measured accelerations is well established, the modal analysis begins by using acceleration measurements to generate a reference. Figure 4 (a) shows the time history of the acceleration at point 3, Figure 4 (b) shows the corresponding spectrum, where the two peaks are at 1.38 Hz and 9.42 Hz, indicating that the first two natural frequencies can be clearly seen. The first two vibration modes determined by NExT-ERA are shown in Figure 2. Figure 5 are incorporated herein by reference.

[0162] In the test, the ROI of point 3 was selected before displacement extraction, as shown in Figure 6 The target length in the frame is 22 mm, corresponding to 118 pixels, so SF = 0.01864 mm / pixel. The time history of pixel displacement is then extracted using template matching measurement technology. The displacement time history at point 3 when the random excitation output voltage is 900 mV is shown as follows: Figure 7 As shown, from Figure 7 In (a), it can be seen that the displacement amplitude is close to 0.2 pixels, and the corresponding spectrum is as follows Figure 7As shown in (b), the two peaks at 1.41Hz and 9.39Hz represent the first two natural frequencies. Figure 4 The peaks in (b) match well.

[0163] In contrast, the displacement of point 2 is as follows Figure 8 As shown, Figure 8 (a) shows a duration of 25 s, Figure 8 (b) shows a duration of 4 s. Overall, the amplitude is about 0.1 pixel, almost half of that of point 3, because point 2 is closer to the fixed end. Figure 8 (b) It can be seen that the displacement curve is jagged, indicating that the displacement resolution is poor. Figure 9 As shown, frequency analysis of the small displacement at point 2 reveals a dominant frequency of 9.39 Hz (the second-order natural frequency). However, in addition to the peak corresponding to the fundamental natural frequency, there are two other peaks caused by noise. The vibration response is masked by camera noise, resulting in a jagged signal and a relatively poor signal-to-noise ratio, making it more challenging to identify the mode shape.

[0164] Figure 10 The displacement and corresponding spectrum for point 2 from the first series of tests are shown. Five excitation output voltage values (300, 600, 900, 1200, and 1500) were selected, while the focal length and object distance remained constant at 35 mm and 160 cm. As expected, as the excitation power increases, the displacement amplitude increases, and the signal-to-noise ratio of the extracted displacement also increases. Consequently, it becomes nearly impossible to calculate acceleration using central differencing for small displacements.

[0165] Since the natural frequency can be obtained by picking the peak from the spectrum, the proposed frequency domain operator can be used to reconstruct the displacement and acceleration. Taking the displacement at point 2 with an excitation output voltage of 600mV as an example, the acceleration obtained by applying the central difference to the original displacement and the reconstructed displacement is compared, as shown in Figure 2. Figure 11 It is observed that the former has a clear sawtooth pattern with extremely high amplitude, indicating a large numerical error, while the latter looks more reasonable. Figure 12 express Figure 11 The spectra of the two accelerations are shown in . It can be observed that the SNR of the reconstructed acceleration is higher than that of the directly calculated acceleration because the frequency components close to the natural frequency are well preserved while the other frequency components are significantly reduced.

[0166] The mode shapes were then evaluated using the NExT-ERA method. Due to the limited field of view of the camera, a mode shape assembly method was employed. For this study, the five points were divided into four groups of two points each. Figure 13The modal vibration shapes identified by the original displacement extracted by the template matching method, the acceleration directly calculated by the central difference, and the acceleration reconstructed by the proposed method were compared. It was observed that when the original displacement was used, the vibration shape could only be identified when the excitation output voltage was above 1200mV, while it could not be identified when the excitation output voltage was between 300 and 900mV. When the directly calculated acceleration was used, the vibration shape could not be identified regardless of the excitation output voltage, which once again proves that the acceleration obtained by directly applying the central difference to the original displacement extracted by the template matching method is not accurate enough. Finally, when the acceleration was reconstructed using the method proposed in the present invention, the first two modal vibration shapes could be accurately evaluated even if the excitation output voltage was as low as 300mV (corresponding to a displacement amplitude of 0.01mm).

[0167] In addition, the relative error of the frequency and the modal assurance criterion (MAC) value are used to quantitatively evaluate the performance of the proposed method. The reference frequency f obtained by the traditional method is r The relative error between the natural frequency f identified by the method of the present invention is defined as follows:

[0168]

[0169] The MAC value is defined as the similarity between the mode shapes identified using the traditional method of directly measuring acceleration and the mode shapes identified using the non-contact method of reconstructing acceleration, and is expressed as:

[0170]

[0171] Where X is the position of the measurement point, φ r is the reference mode shape, and φ is the mode shape identified by the method proposed in this invention. A MAC value close to 1 indicates high similarity, so the identified mode shape is accurate, while a value close to 0 indicates low similarity, and the identified mode shape may be incorrect.

[0172] Table 1. Relative errors and MAC values of natural frequencies and vibration modes identified by the signal reconstruction method

[0173]

[0174] Table 1 shows the relative errors and MAC values for the frequencies and mode shapes identified using the proposed method. The relative errors are generally less than 1%, with a maximum relative error of 2.9%, indicating that the frequencies can be accurately assessed. The MAC values are all greater than 0.95, demonstrating that the method performs well in extracting the mode shapes of structures with small-amplitude vibrations due to environmental excitation.

[0175] Camera settings also affect measurement results. Here, we explore the effects of focal length and object distance on measurement results. Four cases are considered: f = 12 mm, u = 100 cm, f = 12 mm, u = 160 cm, f = 12 mm, u = 200 cm, and f = 35 mm, u = 200 cm. Other conditions remain unchanged: resolution 2448 × 2048, frame rate 50 frames per second, aperture 2.8, and excitation output voltage 1500 mV. As the camera's focal length and object distance change, the number of pixels occupied by the target in the image and the magnitude of the displacement in the image also change. Figure 14 The ROI of point 2 in the image with different camera settings is shown.

[0176] Table 2. Measurement results at different points under different camera parameters

[0177]

[0178]

[0179] Table 2 summarizes the displacement amplitudes extracted using the template matching method and the frequencies obtained by peak-picking from the spectrum. For example, with a focal length of 12 mm and an object distance of 160 cm, the displacement amplitudes at points 2, 3, 4, and 5 are 0.01, 0.03, 0.04, and 0.07, respectively. The displacement amplitude at point 1 is zero because it is located at the fixed end. As expected, the amplitude at point 5 is the largest, and the amplitude at point 1 is the smallest. It can be seen that frequencies cannot be identified by peak-picking from the displacement spectra at points 2 and 3. The primary reason is that the displacement amplitudes are very small and the resolution is very limited. High-order frequencies are particularly susceptible to noise, making their identification difficult. Furthermore, when the object distance remains constant, the displacement amplitude at the same point increases with increasing focal length; when the object distance remains constant, the displacement amplitude at the same point increases with decreasing focal length. Therefore, the accuracy of visual measurement can be improved by reducing the object distance and extending the focal length.

[0180] The vibration shapes extracted from different dynamic responses under different camera settings are as follows: Figure 15 As shown. When the original displacements extracted by the template matching method are used, the modal vibration shapes can be identified if f = 12 mm, u = 100 cm, f = 35 mm, u = 200 cm. When the acceleration calculated from the original displacement by central difference is used, the modal vibration shapes cannot be obtained for these four camera settings. When the acceleration reconstructed by the proposed method is used, the first two modal vibration shapes can be identified for all camera settings, verifying that the proposed method can perform well under various camera settings. In addition, Table 3 shows the relative errors and MAC values of the identified natural frequencies and modal vibration shapes. It can be seen that the maximum relative error is 3.6% and the MAC values are all greater than 0.95, verifying that the proposed method performs well in identifying the natural frequencies and vibration shapes of structures with low amplitude vibrations under various camera settings.

[0181] Table 3. Modal parameter identification results of videos compared with reference results of accelerometers with different camera parameters

[0182]

[0183] Further references Figure 16 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a signal reconstruction device for visual structural vibration mode recognition under environmental excitation. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0184] The present invention provides a signal reconstruction device for visually identifying structural vibration modes under environmental excitation, comprising:

[0185] The displacement signal extraction module 1 is configured to obtain a source image of the structure to be measured, and extract the displacement signal of the region of interest of the structure to be measured by a template matching method combined with sub-pixel technology based on the region of interest of several frames of the source image;

[0186] The signal reconstruction module 2 is configured to reconstruct the displacement signal to obtain a reconstructed displacement signal, and obtain a reconstructed acceleration signal according to the reconstructed displacement signal;

[0187] The local mode shape determination module 3 is configured to obtain the mode shape of the region of interest based on the acceleration signal using a natural excitation technique combined with an eigensystem implementation algorithm;

[0188] The splicing module 4 is configured to repeatedly execute the displacement signal extraction module to the local vibration mode determination module to obtain the vibration modes of all regions of interest in the source image of the structure to be measured, and splice them to obtain the vibration modes of the structure to be measured.

[0189] Reference below Figure 17 , which shows an electronic device (eg Figure 1 Schematic diagram of the structure of a computer device 1700 (server or terminal device shown). Figure 17 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0190] like Figure 17As shown, the computer device 1700 includes a central processing unit (CPU) 1701 and a graphics processing unit (GPU) 1702, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1703 or the program loaded from the storage part 1709 to the random access memory (RAM) 1704. Various programs and data required for the operation of the device 1700 are also stored in the RAM 1704. The CPU 1701, GPU 1702, ROM 1703 and RAM 1704 are connected to each other via a bus 1705. An input / output (I / O) interface 1706 is also connected to the bus 1705.

[0191] The following components are connected to the I / O interface 1706: an input section 1707 including a keyboard, a mouse, and the like; an output section 1708 including, for example, a liquid crystal display (LCD), and a speaker; a storage section 1709 including, for example, a hard disk; and a communication section 1710 including, for example, a network interface card such as a LAN card or a modem. The communication section 1710 performs communication processing via a network such as the Internet. A drive 1711 may also be connected to the I / O interface 1706 as needed. Removable media 1712, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1711 as needed, so that computer programs read therefrom can be installed into the storage section 1709 as needed.

[0192] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 1710, and / or installed from a removable medium 1712. When the computer program is executed by the central processing unit (CPU) 1701 and the graphics processing unit (GPU) 1702, the above-mentioned functions defined in the method of the present application are executed.

[0193] It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable medium, or any combination thereof. Computer-readable media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, apparatuses, or components, or any combination thereof. More specific examples of computer-readable media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device, apparatus, or component. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution apparatus, device, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0194] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0195] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based device that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0196] The modules involved in the embodiments described in this application may be implemented in software or hardware, and may also be set in a processor.

[0197] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or it may exist independently and not be assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device is caused to: obtain a source image of the structure to be measured, extract a displacement signal of the region of interest of the structure to be measured based on the region of interest of several frames of source images by a template matching method combined with sub-pixel technology; reconstruct the displacement signal to obtain a reconstructed displacement signal, and obtain a reconstructed acceleration signal based on the reconstructed displacement signal; obtain the mode shape of the region of interest based on the acceleration signal using a natural excitation technology combined with an intrinsic system implementation algorithm; repeat the above steps to obtain the mode shapes of all regions of interest in the source image of the structure to be measured, and splice them to obtain the mode shape of the structure to be measured.

[0198] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A signal reconstruction method for visually identifying structural vibration modes under environmental excitation, characterized in that: The following steps are involved: S1, obtaining a source image of the structure to be measured, and extracting a displacement signal of the region of interest of the structure to be measured by a template matching method combined with sub-pixel technology based on the region of interest of the source image of several frames; S2, reconstructing the displacement signal to obtain a reconstructed displacement signal, and obtaining a reconstructed acceleration signal according to the reconstructed displacement signal; the step S2 specifically includes: Considering the sub-pixel displacement signal y(t) extracted at a certain point in the region of interest of the structure to be measured, its Fourier transform is expressed as: Among them, F y (ω) is the spectrum of the displacement signal. Based on the above formula, a frequency domain operator is proposed and defined as follows: Among them, β represents the proportional factor, ω m =2πf m is the mth angular frequency of the structure, Δω is the half bandwidth; The time history of the displacements is then reconstructed by using an inverse Fourier transform to generate the following equation: in, The reconstructed displacement signal is obtained by applying the central difference to the reconstructed displacement, and the reconstructed acceleration signal is expressed as: Among them, the reconstructed acceleration signal is Δt is the corresponding time interval in the time domain. When the Fourier transform is applied to When , we get: S3, obtaining the vibration mode of the region of interest using a natural excitation technique combined with an eigensystem implementation algorithm according to the acceleration signal; S4, repeating steps S1-S3 to obtain the mode shapes of all regions of interest in the source image of the structure to be measured, and splicing them to obtain the mode shapes of the structure to be measured.

2. The signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation according to claim 1 is characterized in that: The step S1 includes: Before measurement, camera calibration is performed to determine the relationship between pixel coordinates and physical coordinates. When the image plane is parallel to the object surface, the scale factor method is used for camera calibration. The scale factor on the x-axis can be expressed as follows: Where d is the pixel length of the structure in the image, D is the real length of the structure, z is the distance between the camera and the structure, and f is the focal length of the camera.

3. The signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation according to claim 1 is characterized in that: The source image includes several regions of interest, and each region of interest includes a template image.

4. The signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation according to claim 1, characterized in that: The step S1 specifically includes: A region of interest containing a template image is determined in the source image, the template image is moved pixel by pixel to determine a matching region, the template image is compared with the region of interest, the frequency domain cross-correlation between the template image and the region of interest is calculated, the position of the template image in the source image is determined, and a displacement signal is obtained based on the pixel difference between the template image and two adjacent frames of the source image.

5. The signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation according to claim 4, characterized in that: Calculating the frequency domain cross-correlation between the template image and the region of interest, determining the position of the template image in the source image, and obtaining a displacement signal according to a pixel difference between two adjacent frames of the source image of the template image specifically includes: Consider a template image I(x,y) and a region of interest T(x,y) of the same dimension M×N. Apply Fourier transform to I(x,y) and T(x,y). The cross-correlation between the two is expressed as: Wherein, Δx and Δy are the coordinate offsets in the x and y directions, P(u,v) is the discrete Fourier transform of the template image, Q(u,v) is the discrete Fourier transform of the source image, * represents the complex conjugate; u and v represent the abscissa and ordinate of the image in the frequency domain after the Fourier transform, respectively; x' represents the abscissa of the image in the spatial domain after the offset Δx; y' represents the ordinate of the image in the spatial domain after the offset Δy; M is the length of the template image, and N is the width of the template image; By searching the cross-correlation F corr The peak value of the pixel-level coordinates (x p ,y p ), calculate the pixel-level coordinates (x p ,y p ) by searching the cross-correlation F corr The maximum value of the sub-pixel coordinates (x sp ,y sp ); The first frame source image is used as the reference frame, and the sub-pixel coordinates of the reference frame are (x sp,1 ,y sp,1 ), the sub-pixel coordinates of the template image in the region of interest of each new frame source image are (x sp,i ,y sp,i ), the sub-pixel displacement signal of the region of interest of the structure to be measured is expressed as: y(t)=(x sp,i ,y sp,i )-(x sp,1 ,y sp,1 ).

6. The signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation according to claim 1, characterized in that: The step S3 specifically includes: In the discrete time domain, the dynamics of a linear and time-invariant dynamical system with n degrees of freedom is described by the state-space equations as follows: p(k+1)=A d p(k)+B d u(k); q(k)=C d p(k)+D d u(k); Among them, k represents the kth discrete time step, A d ∈R 2n×2n 、B d ∈R 2n×l 、C d ∈R m×2n and D d ∈R m×l They are the state matrix, input matrix, output matrix and transfer matrix, respectively, where the state matrix, input matrix and transfer matrix are known quantities related to the mass stiffness and damping of the structure; p(k)∈R 2n×l ,q(k)∈R m×l and u(k)∈R l×1 They are 2n-dimensional input vector, m-dimensional output vector, and l-dimensional excitation vector respectively; l represents the number of inputs, m represents the number of outputs, and 2n represents the order of the state matrix; the environmental excitation is used as the input vector p(k), and the reconstructed acceleration signal is used as the output vector q(k), and the output matrix C is obtained. d ; When the output vector q(k) is specified as an impulse response, the following Hankel matrix is formed: Where α = 1, 2, 3…, β = 1, 2, 3…, each element in the Hankel matrix is an m×n dimensional matrix, so the order of the Hankel matrix is αm×βn; Obtain similarity matrix by two Hankel matrices with the same time step The calculation is as follows: Among them, V k-1 、U k-1 and Γ k-1 Satisfies the singular value decomposition of the Hankel matrix, that is: Similarity Matrix The eigenvalue decomposition of can be used to calculate the natural frequencies as follows: Among them, Λ d is a diagonal matrix, Λ d yes The eigenvalue matrix, Ψ is the eigenvector matrix, ω d,n , Denote the nth order angular frequency and its complex conjugate respectively, and the mth natural frequency f of the structure to be measured is obtained using the following equation m : The sampling frequency is f s , the eigenvalue matrix is obtained through the natural frequency, and the eigenvector matrix Ψ is calculated according to the eigenvalue matrix; The mode shapes for the region of interest are generated using the following equation: Φ=C d Ps.

7. A signal reconstruction device for visual-based structural vibration mode recognition under environmental excitation, using the signal reconstruction method for visual-based structural vibration mode recognition under environmental excitation according to any one of claims 1 to 6, characterized in that: include: a displacement signal extraction module configured to obtain a source image of the structure to be measured, and extract a displacement signal of the region of interest of the structure to be measured based on the region of interest of the source image in several frames by using a template matching method combined with sub-pixel technology; a signal reconstruction module configured to reconstruct the displacement signal to obtain a reconstructed displacement signal, and obtain a reconstructed acceleration signal according to the reconstructed displacement signal; A local mode shape determination module is configured to obtain the mode shape of the region of interest using a natural excitation technique combined with an eigensystem implementation algorithm according to the acceleration signal; The splicing module is configured to repeatedly execute the displacement signal extraction module to the local vibration shape determination module to obtain the vibration shapes of all the regions of interest in the source image of the structure to be measured, and splice them to obtain the vibration shapes of the structure to be measured.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images