Structural modal identification method and device based on undersampled video

Abstract

The invention discloses a structure modal identification method and device based on an undersampled video, and the method comprises the steps: obtaining a vibration video of a structure, and obtaining a speed signal and a displacement signal of the vibration of the structure through employing an LK optical flow method and a template matching method based on the vibration video; performing displacement interpolation based on the speed signal to obtain a structure vibration displacement signal which is increased by three times; processing the newly obtained structure vibration displacement signal by adopting a blind source separation algorithm to obtain a structure vibration mode and a non-uniform modal coordinate independent sample; and processing the non-uniform modal coordinate independent sample through a compressed sensing algorithm, recovering to obtain a displacement signal in a uniform state, and performing fast Fourier transform on the displacement signal in the uniform state to obtain the frequency of structural vibration. According to the method, the frequency and the vibration mode of the structure can be obtained under the condition of undersampling shooting, and no priori knowledge is needed, so that the health detection of the structure is realized.

Description

Technical Field

[0001] The present invention relates to the field of structural vibration analysis, and in particular to a structural mode recognition method and device based on under-sampling video. Background Art

[0002] Improving the spatial and temporal resolution of vibration measurements and modal analysis can significantly benefit dynamic modeling, analysis, and health monitoring of structures. In experimental or operational modal analysis, higher modes contain local structural features that can improve damage localization and the construction and updating of modal-based structural dynamic models, but may be beyond the measurement frequency range. In general, the resolution of vibration measurements can be improved through hardware enhancement. Traditional vibration measurement sensors such as accelerometers have high-frequency sampling capabilities; however, they are discrete point sensors that only provide sparse, low spatial sensing resolution measurements, while densely deployed equipment to achieve high spatial resolution is expensive and can result in mass loading effects and modifications to the structure surface.

[0003] Non-contact measurement methods, such as digital cameras, are relatively low cost, flexible, and offer high spatial resolution and the ability to perform simultaneous measurements. However, the sampling frequency of most economical digital cameras is limited to 30-60Hz, while high-speed cameras for higher frequency vibration measurements are extremely expensive. Summary of the invention

[0004] The purpose of this application is to propose a structural modal recognition method and device based on under-sampling video to address the above-mentioned technical problems, which can extract high-order modes of structural vibration when the sampling rate is 30-60Hz.

[0005] In a first aspect, the present invention provides a structural modality recognition method based on undersampled video, comprising the following steps:

[0006] Obtain a vibration video of the structure, and use the LK optical flow method and template matching method based on the vibration video to obtain the velocity signal and displacement signal of the structural vibration;

[0007] Interpolation is performed based on the velocity signal and the displacement signal to obtain a reconstructed structural vibration displacement signal;

[0008] The reconstructed structural vibration displacement signal is processed by blind source separation algorithm to obtain independent samples of structural vibration shapes and non-uniform modal coordinates.

[0009] The non-uniform modal coordinate independent samples are processed by the compressed sensing algorithm to restore the displacement signal in the uniform state, and the frequency of the structural vibration is obtained by using the fast Fourier transform of the displacement signal in the uniform state.

[0010] Preferably, the LK optical flow method and the template matching method are respectively used based on the vibration video to obtain the velocity signal and the displacement signal of the structural vibration, specifically including:

[0011] I(x, y, t) is the grayscale value of the pixel at the coordinate position (x, y) in the video frame at time t in the vibration video. Assuming that the pixel moves to the coordinate position (x+Δx, y+Δy) after time Δt, the following formula is obtained according to the optical flow brightness consistency constraint:

[0012] I(x+Δx,y+Δy,t+Δt)=I(x,y,t);

[0013] Wherein, Δx represents the distance that the pixel point moves along the x-axis within the time Δt, and Δy represents the distance that the pixel point moves along the y-axis within the time Δt;

[0014] Expand the above equation using Taylor's formula, and ignore the terms after the second order based on the assumption of small displacement motion, to obtain the following equation:

[0015]

[0016] Eliminate the same terms I(x,y,t) on both sides of the above equation and divide by Δt to simplify it to:

[0017]

[0018] make The optical flow constraint equation is obtained as shown below:

[0019]

[0020] Among them, (u,v) represents the optical flow value, u and v are the optical flow speed in the x-axis direction and the optical flow speed in the y-axis direction respectively. and Represent the first-order partial derivatives of the gray value I with respect to x, y and t respectively;

[0021] According to the spatial consistency hypothesis, one of the pixels in the video frame of the vibration video is taken as the central pixel, and the movement speed of the central pixel is calculated by establishing a neighborhood pixel system equation. If the neighborhood size of the central pixel is n×n, the neighborhood pixel system equation is:

[0022]

[0023] Representing the matrix in the neighborhood pixel system equation with letters, we get the following formula:

[0024] Bd = b;

[0025] in,

[0026] Multiply both sides of the above formula by B T , we get the following formula:

[0027] (B T B)d=B T b;

[0028] Where T represents transpose;

[0029] When (B T B) When reversible, the solution of the neighborhood pixel system equation is:

[0030]

[0031] The relationship between the velocity signal V and the optical flow velocity v in the y-axis direction is determined by the motion relationship between the optical flow field of the structural vibration and the motion field, as shown in the following formula:

[0032]

[0033] Among them, g is the shortest distance between the structure and the camera, and f is the focal length of the lens;

[0034] The relationship between pixel coordinates and physical coordinates is determined by camera calibration. When the image plane is parallel to the object surface, the scale factor method is used for camera calibration to obtain the scale factor.

[0035] The video frame of the vibration video is used as the source image, the sensing area containing the template image is determined in the source image, the frequency domain cross-correlation between the template image and the region of interest is calculated, the position of the template image in the source image is determined, and the pixel difference of the template image between two adjacent frames of the source image is multiplied by the scale factor to obtain the displacement signal.

[0036] Preferably, interpolation is performed based on the velocity signal and the displacement signal to obtain a reconstructed structural vibration displacement signal, which specifically includes:

[0037] The reconstructed structural vibration displacement signal is calculated using the following formula:

[0038] z(t i )=Y(t i );

[0039]

[0040] z(t i -dt)=Y(t i )-dtV i ;

[0041] z(t i +dt)=Y(t i )+dtV i ;

[0042] Among them, Y(t i ) indicates t i The displacement signal at the moment, Y(t i+1 ) indicates t i+1 The displacement signal at the moment, V i Indicates t i The speed signal at the moment, V i+1 Indicates t i+1 The speed response of the moment, i=1,2,…,n, f s Indicates the sampling rate of the camera, The z(t i ), z(t i -dt) and z(t i +dt) are combined in time sequence to obtain the reconstructed structural vibration displacement signal z(t).

[0043] Preferably, a blind source separation algorithm is used to process the reconstructed structural vibration displacement signal to obtain independent samples of structural vibration shapes and non-uniform modal coordinates, specifically including:

[0044] The structural vibration is regarded as a linear time-invariant system with s degrees of freedom, and its vibration differential equation is expressed as:

[0045]

[0046] Among them, x(t), denote the displacement signal, velocity signal and acceleration signal of the linear time-invariant system, respectively. M, C and K are the mass, damping and stiffness matrices, respectively, and are assumed to be real and symmetric.

[0047] The reconstructed structural vibration displacement signal is input into the second-order blind source separation algorithm to obtain the modal vibration matrix A. For a small damping structure, the reconstructed structural vibration displacement signal is expressed in the modal coordinates as follows:

[0048]

[0049] Where A is the s-order mode shape vector A h The modal vibration matrix q(t) is composed of the modal response signals q h (t) composed of non-uniform modal coordinate independent samples, h = 1, 2, ..., s, q (t) = [q 1 (t),q 2 (t),…,q s (t)] T ;

[0050] The non-uniform modal coordinate independent sample q(t) is calculated by the above formula;

[0051] The modal vibration matrix A is judged for positive and negative signs and regularized to obtain the corresponding structural vibration shape.

[0052] Preferably, the non-uniform modal coordinate independent samples are processed by a compressed sensing algorithm to restore the displacement signal in a uniform state, which specifically includes:

[0053] The following formula is used to process the non-uniform modal coordinate independent samples:

[0054] ω=argmin||ω|| 1 ,stH(t)=AΨω;

[0055] Where A is the mode shape matrix, is the discrete Fourier basis, q(t) is the independent sample of the non-uniform modal coordinates, H(t) represents the displacement signal under the uniform state, ω=e -2πk / N is the Nth primitive root of unity, k 2 =-1,

[0056]

[0057] As a preferred embodiment, the vibration video is acquired by using a camera, and the sampling rate of the camera is f s The range is 30-60Hz.

[0058] In a second aspect, the present invention provides a structural modality recognition device based on under-sampled video, comprising:

[0059] A signal acquisition module is configured to acquire a vibration video of the structure, and obtain a velocity signal and a displacement signal of the structural vibration by using an LK optical flow method and a template matching method based on the vibration video;

[0060] A data reconstruction module is configured to perform interpolation based on the velocity signal and the displacement signal to obtain a reconstructed structural vibration displacement signal;

[0061] A signal processing module is configured to process the reconstructed structural vibration displacement signal using a blind source separation algorithm to obtain independent samples of structural vibration shapes and non-uniform modal coordinates;

[0062] The frequency calculation module is configured to process the non-uniform modal coordinate independent samples through a compressed sensing algorithm, recover the displacement signal under a uniform state, and use fast Fourier transform on the displacement signal under a uniform state to obtain the frequency of structural vibration.

[0063] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0064] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0065] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] (1) The structural modal recognition method based on under-sampled video proposed in the present invention can obtain the high-order modes of the structure under under-sampled shooting without any prior knowledge, thereby realizing structural health detection.

[0068] (2) The structural modal recognition method based on under-sampled video proposed in the present invention does not require the use of a high-speed camera, but can be implemented using only a conventional camera, which can greatly reduce the cost of testing.

[0069] (3) The structural modal recognition method based on under-sampled video proposed in the present invention only requires one sampling, thereby improving operability. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 A schematic flow chart of a structural modality recognition method based on under-sampled video according to an embodiment of the present application;

[0072] Figure 2 Graphs showing structural vibration mode recognition results of the structural modal recognition method based on under-sampling video according to an embodiment of the present application, wherein (a) is a first-order vibration mode; (b) is a second-order vibration mode; (c) is a third-order vibration mode;

[0073] Figure 3Graphs showing frequency recognition results of a structural modal recognition method based on under-sampling video according to an embodiment of the present application, wherein (a) is a frequency recognition result of a control group; (b) is a frequency recognition result of under-sampling; and (c) is a frequency recognition result after applying the present invention.

[0074] Figure 4 Schematic diagram of a structural modality recognition device based on under-sampled video according to an embodiment of the present application;

[0075] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] Figure 1 The embodiment of the present application provides a structural modality recognition method based on under-sampled video, comprising the following steps:

[0078] S1, obtain the vibration video of the structure, and use the LK optical flow method and template matching method based on the vibration video to obtain the velocity signal and displacement signal of the structural vibration.

[0079] In a specific embodiment, the vibration video is acquired by using a camera, and the sampling rate of the camera is f s The range is 30-60Hz.

[0080] In a specific embodiment, the velocity signal and displacement signal of the structural vibration are obtained by using the LK optical flow method and the template matching method based on the vibration video, specifically including:

[0081] I(x, y, t) is the grayscale value of the pixel at the coordinate position (x, y) in the video frame at time t in the vibration video. Assuming that the pixel moves to the coordinate position (x+Δx, y+Δy) after time Δt, the following formula is obtained according to the optical flow brightness consistency constraint:

[0082] I(x+Δx,y+Δy,t+Δt)=I(x,y,t);

[0083] Wherein, Δx represents the distance that the pixel point moves along the x-axis within the time Δt, and Δy represents the distance that the pixel point moves along the y-axis within the time Δt;

[0084] Expand the above equation using Taylor's formula, and ignore the terms after the second order based on the assumption of small displacement motion, to obtain the following equation:

[0085]

[0086] Eliminate the same terms I(x,y,t) on both sides of the above equation and divide by Δt to simplify it to:

[0087]

[0088] make The optical flow constraint equation is obtained as shown below:

[0089]

[0090] Among them, (u,v) represents the optical flow value, u and v are the optical flow speed in the x-axis direction and the optical flow speed in the y-axis direction respectively. and Represent the first-order partial derivatives of the gray value I with respect to x, y and t respectively;

[0091] According to the spatial consistency hypothesis, one of the pixels in the video frame of the vibration video is taken as the central pixel, and the movement speed of the central pixel is calculated by establishing a neighborhood pixel system equation. If the neighborhood size of the central pixel is n×n, the neighborhood pixel system equation is:

[0092]

[0093] Representing the matrix in the neighborhood pixel system equation with letters, we get the following formula:

[0094] Bd = b;

[0095] in,

[0096] Multiply both sides of the above formula by B T , we get the following formula:

[0097] (B T B)d=B T b;

[0098] Where T represents transpose;

[0099] When (B T B) When reversible, the solution of the neighborhood pixel system equation is:

[0100]

[0101] The relationship between the velocity signal V and the optical flow velocity v in the y-axis direction is determined by the motion relationship between the optical flow field of the structural vibration and the motion field, as shown in the following formula:

[0102]

[0103] Among them, g is the shortest distance between the structure and the camera, and f is the focal length of the lens;

[0104] The relationship between pixel coordinates and physical coordinates is determined by camera calibration. When the image plane is parallel to the object surface, the scale factor method is used for camera calibration to obtain the scale factor.

[0105] The video frame of the vibration video is used as the source image, the sensing area containing the template image is determined in the source image, the frequency domain cross-correlation between the template image and the region of interest is calculated, the position of the template image in the source image is determined, and the pixel difference of the template image between two adjacent frames of the source image is multiplied by the scale factor to obtain the displacement signal.

[0106] Specifically, a vibration video of the structure is collected, and feature points in the vibration video are selected as sampling points. The vibration video collection process in the embodiment of the present application is shot using an industrial camera MV-CH250-90YM-C-NF. The vibration video of the structure captured is respectively subjected to the LK optical flow method and the template matching method to obtain the structural velocity response and displacement response. The sampling rate of the camera is f s The motion projection of an object on an image reflects the movement of pixels, and the instantaneous movement speed of the pixels is the optical flow. The optical flow method uses the changes in the pixels in the image sequence in the time domain and the correlation between adjacent frames to find the correspondence between the previous frame and the current frame, and calculates the motion information of objects between adjacent frames.

[0107] The specific process of obtaining the displacement signal using the template matching method is as follows:

[0108] First, the relationship between pixel coordinates and physical coordinates is determined by camera calibration. When the image plane is parallel to the object surface, the scale factor method is used for camera calibration. The scale factor is expressed as:

[0109]

[0110] Where SF is the scale factor, D is the actual length of the target, and d is the pixel length of the target in the image;

[0111] The source image contains several regions of interest, and each source image contains a template image. Once the camera calibration is completed, the source image can be collected, and each frame of the collected vibration video is used as the source image, and then the target can be tracked. The template image is moved pixel by pixel to determine the matching area, the template image is compared with each region of interest (ROI) in the source image, the frequency domain cross-correlation between the template image and each region of interest is calculated, the position of the template image in the source image is determined, and the displacement signal is obtained according to the pixel difference between the template image and two adjacent frames of the source image. The embodiment of the present application adopts the cross-correlation standard for similarity evaluation.

[0112] Consider a template image J(x,y) and a region of interest T(x,y) with the same dimension M×L. Apply Fourier transform to J(x,y) and T(x,y). The frequency domain cross-correlation between the two is expressed as:

[0113]

[0114] Wherein, Δx and Δy represent the coordinate offset of the template image in the x and y directions respectively, P(u,v) is the discrete Fourier transform (DFT) of the template image, P(u,v) is the discrete Fourier transform of the template image, K(u,v) is the discrete Fourier transform of the source image, * represents the complex conjugate; u and v represent the horizontal and vertical coordinates of the image in the frequency domain after the Fourier transform respectively; x' represents the horizontal coordinate of the template image in the spatial domain after the offset of Δx; y' represents the vertical coordinate of the template image in the spatial domain after the offset of Δy; M is the length of the template image, and L is the width of the template image;

[0115] The pixel-level coordinates (x o,e ,y o,e ). The first source image is used as the reference frame, and the pixel coordinates of the reference frame are (x o,1 ,y o,1 ), the pixel coordinates of the template image in the region of interest of each frame of the source image are (x o,e ,y o,e ), then the pixel-level displacement signal of the region of interest of the structure to be measured is expressed as:

[0116] y(t)=(x p,1 ,y p,1 )-(x p,e ,y p,e );

[0117] Then the displacement signal of the structural vibration is:

[0118] Y(t)=SF·y(t);

[0119] in,

[0120] S2, interpolating based on the velocity signal and the displacement signal to obtain a reconstructed structural vibration displacement signal.

[0121] In a specific embodiment, step S2 specifically includes:

[0122] The reconstructed structural vibration displacement signal is calculated using the following formula:

[0123] z(t i )=Y(t i );

[0124]

[0125] z(t i -dt)=Y(t i )-dtV i ;

[0126] z(t i +dt)=Y(t i )+dtV i ;

[0127] Among them, Y(t i ) indicates t i The displacement signal at the moment, Y(t i+1 ) indicates t i+1 The displacement signal at the moment, V i Indicates t i The speed signal at the moment, V i+1 Indicates t i+1 The speed response of the moment, i=1,2,…,n, f s Indicates the sampling rate of the camera, The z(t i ), z(t i -dt) and z(t i +dt) are combined in time sequence to obtain the reconstructed structural vibration displacement signal z(t).

[0128] Specifically, interpolation is performed based on the velocity signal and the displacement signal of the structural vibration to obtain a reconstructed structural vibration displacement signal.

[0129] S3, the reconstructed structural vibration displacement signal is processed by blind source separation algorithm to obtain independent samples of structural vibration shapes and non-uniform modal coordinates.

[0130] In a specific embodiment, step S3 specifically includes:

[0131] The structural vibration is regarded as a linear time-invariant system with s degrees of freedom, and its vibration differential equation is expressed as:

[0132]

[0133] Among them, x(t), denote the displacement signal, velocity signal and acceleration signal of the linear time-invariant system, respectively. M, C and K are the mass, damping and stiffness matrices, respectively, and are assumed to be real and symmetric.

[0134] The reconstructed structural vibration displacement signal is input into the second-order blind source separation algorithm to obtain the modal vibration matrix A. For a small damping structure, the reconstructed structural vibration displacement signal is expressed in the modal coordinates as follows:

[0135]

[0136] Where A is the s-order mode shape vector A h The modal vibration matrix q(t) is composed of the modal response signals q h (t) composed of non-uniform modal coordinate independent samples, h = 1, 2, ..., s, q (t) = [q 1 (t),q 2 (t),…,q s (t)] T ;

[0137] The non-uniform modal coordinate independent sample q(t) is calculated by the above formula;

[0138] The modal vibration matrix A is judged for positive and negative signs and regularized to obtain the corresponding structural vibration shape.

[0139] Specifically, the reconstructed structural vibration displacement signal is further processed using a blind source separation algorithm to obtain independent samples q(t) of structural vibration shapes and non-uniform modal coordinates.

[0140] S4, the non-uniform modal coordinate independent samples are processed by the compressed sensing algorithm to restore the displacement signal in the uniform state, and the frequency of the structural vibration is obtained by using the fast Fourier transform of the displacement signal in the uniform state.

[0141] In a specific embodiment, step S4 specifically includes:

[0142] The following formula is used to process the non-uniform modal coordinate independent samples:

[0143] ω=argmin||ω|| 1 ,stH(t)=AΨω;

[0144] Where A is the mode shape matrix, is the discrete Fourier basis, q(t) is the independent sample of the non-uniform modal coordinates, H(t) represents the displacement signal under the uniform state, ω=e -2πk / N is the Nth primitive root of unity, k 2 =-1,

[0145]

[0146] Specifically, firstly, compressed sensing is used to process the non-uniform modal coordinate samples q(t) to restore the displacement response H(t) under the uniform signal. Secondly, the fast Fourier transform (FFT) is used on H(t) to obtain the frequency f of the structural vibration.

[0147] The following is further described with specific examples.

[0148] The model used in this embodiment is a cantilever beam structure with a rectangular cross-section. Its material is aluminum with a density of 2700 kg / m 3 , the elastic modulus is 6.9×10 11 Pa, Poisson's ratio is 0.35, its total length is 1000mm, height is 20mm, thickness is 2mm. 10 sampling points are evenly selected from the beam at intervals of 111.1mm, and the sampling rate is 50hz. The results of the 700hz sampling rate are used as the control group.

[0149] refer to Figure 2 The treatment group that did not use this method could only identify the first-order vibration mode well, the second-order vibration mode was not well identified, and the third-order vibration mode was completely unidentifiable; after using the present invention, the first three-order vibration modes could be well identified.

[0150] refer to Figure 3 , Figure 3 (a) is the frequency identification result of the control group. The natural frequencies of the structure are 5.2 Hz, 32.3 Hz, and 90.7 Hz. Figure 3 (b) is the under-sampling frequency recognition result, where only the first-order frequency of 5.17 Hz can be recognized; Figure 3 (c) is the frequency recognition result after applying the present invention. The frequencies of the first three orders can be well recognized.

[0151] Further references Figure 4 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a structural modal recognition device based on undersampled video. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0152] The embodiment of the present application provides a structural modality recognition device based on undersampled video, comprising:

[0153] A signal acquisition module 1 is configured to acquire a vibration video of the structure, and obtain a velocity signal and a displacement signal of the structural vibration by using an LK optical flow method and a template matching method based on the vibration video;

[0154] A data reconstruction module 2 is configured to perform interpolation based on the velocity signal and the displacement signal to obtain a reconstructed structural vibration displacement signal;

[0155] The signal processing module 3 is configured to process the reconstructed structural vibration displacement signal using a blind source separation algorithm to obtain independent samples of structural vibration shapes and non-uniform modal coordinates;

[0156] The frequency calculation module 4 is configured to process the non-uniform modal coordinate independent samples through a compressed sensing algorithm, recover the displacement signal in a uniform state, and use fast Fourier transform on the displacement signal in a uniform state to obtain the frequency of the structural vibration.

[0157] Figure 5 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Figure 5 As shown, the electronic device of this embodiment includes: a processor 501 and a memory 502; wherein the memory 502 is used to store computer-executable instructions; the processor 501 is used to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0158] Optionally, the memory 502 may be independent or integrated with the processor 501 .

[0159] When the memory 502 is independently provided, the electronic device further includes a bus 503 for connecting the memory 502 and the processor 501 .

[0160] The embodiment of the present invention further provides a computer storage medium, in which computer execution instructions are stored. When the processor 501 executes the computer execution instructions, the above method is implemented.

[0161] The embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by the processor 501, the above method is implemented.

[0162] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0163] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0164] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The unit formed by the above modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0165] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor 501 to perform some steps of the methods of various embodiments of the present application.

[0166] It should be understood that the processor 501 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor 501 may be any conventional processor 501, etc. The steps of the method disclosed in the invention may be directly embodied in the execution of the hardware processor 501, or may be executed by a combination of hardware and software modules in the processor 501.

[0167] The memory 502 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

[0168] The bus 503 may be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 503 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 503 in the drawings of the present application is not limited to only one bus 503 or one type of bus 503.

[0169] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0170] An exemplary storage medium is coupled to the processor 501, so that the processor 501 can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor 501. The processor 501 and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor 501 and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0171] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A structural modal recognition method based on undersampled video, characterized in that: The following steps are involved: Acquire a vibration video of the structure, and obtain a velocity signal and a displacement signal of the structural vibration by using an LK optical flow method and a template matching method based on the vibration video; Perform interpolation based on the velocity signal and the displacement signal to obtain a reconstructed structural vibration displacement signal; The reconstructed structural vibration displacement signal is processed by blind source separation algorithm to obtain independent samples of structural vibration shapes and non-uniform modal coordinates. The non-uniform modal coordinate independent samples are processed by a compressed sensing algorithm to recover the displacement signal in a uniform state, and the frequency of structural vibration is obtained by using fast Fourier transform on the displacement signal in the uniform state.

2. The structural modal recognition method based on under-sampling video according to claim 1, characterized in that: Based on the vibration video, the LK optical flow method and the template matching method are used to obtain the velocity signal and displacement signal of the structural vibration, specifically including: I(x, y, t) is the grayscale value of the pixel at the coordinate position (x, y) in the video frame at time t in the vibration video. Assuming that the pixel moves to the coordinate position (x+Δx, y+Δy) after time Δt, the following formula is obtained according to the optical flow brightness consistency constraint: I(x+Δx,y+Δy,t+Δt)=I(x,y,t); Wherein, Δx represents the distance that the pixel point moves along the x-axis within the time Δt, and Δy represents the distance that the pixel point moves along the y-axis within the time Δt; Expand the above equation using Taylor's formula, and ignore the terms after the second order based on the assumption of small displacement motion, to obtain the following equation: Eliminate the same terms I(x,y,t) on both sides of the above equation and divide by Δt to simplify it to: make The optical flow constraint equation is obtained as shown below: Among them, (u,v) represents the optical flow value, u and v are the optical flow speed in the x-axis direction and the optical flow speed in the y-axis direction respectively. and Represent the first-order partial derivatives of the gray value I with respect to x, y and t respectively; According to the spatial consistency assumption, one of the pixels in the video frame of the vibration video is taken as the central pixel, and the movement speed of the central pixel is calculated by establishing a neighborhood pixel system equation. If the neighborhood size of the central pixel is n×n, the neighborhood pixel system equation is: The matrix in the neighborhood pixel system equation is represented by letters, and the following formula is obtained: Bd = b; in, Multiply both sides of the above formula by B T , and we get the following formula: (B T B)d=B T b; Where T represents transpose; When (B T B) When reversible, the solution of the neighborhood pixel system equation is: The relationship between the velocity signal V and the optical flow velocity v in the y-axis direction is determined by the motion relationship between the optical flow field of the structural vibration and the motion field, as shown in the following formula: Among them, g is the shortest distance between the structure and the camera, and f is the focal length of the lens; The relationship between pixel coordinates and physical coordinates is determined by camera calibration. When the image plane is parallel to the object surface, the scale factor method is used for camera calibration to obtain the scale factor. The video frame of the vibration video is used as a source image, the sensing area containing the template image is determined in the source image, the frequency domain cross-correlation between the template image and the region of interest is calculated, the position of the template image in the source image is determined, and the pixel difference between two adjacent frames of the template image is multiplied by the scale factor to obtain a displacement signal.

3. The structural modal recognition method based on under-sampling video according to claim 1, characterized in that: Interpolation is performed based on the velocity signal and the displacement signal to obtain a reconstructed structural vibration displacement signal, specifically including: The reconstructed structural vibration displacement signal is calculated using the following formula: z(t i )=Y(t i ); z(t i -dt)=Y(t i )-dtV i ; z(t i +dt)=Y(t i )+dtV i ; Among them, Y(t i ) indicates t i The displacement signal at the moment, Y(t i+1 ) indicates t i+1 The displacement signal at the moment, V i Indicates t i The speed signal at the moment, V i+1 Indicates t i+1 The speed response of the moment, f s Indicates the sampling rate of the camera, The z(t i ), z(t i -dt) and z(t i +dt) are combined in time sequence to obtain the reconstructed structural vibration displacement signal z(t).

4. The structural modal recognition method based on under-sampling video according to claim 1, characterized in that: The reconstructed structural vibration displacement signal is processed by blind source separation algorithm to obtain independent samples of structural vibration shape and non-uniform modal coordinates, including: The structural vibration is regarded as a linear time-invariant system with s degrees of freedom, and its vibration differential equation is expressed as: Among them, x(t), denote the displacement signal, velocity signal and acceleration signal of the linear time-invariant system, respectively. M, C and K are the mass, damping and stiffness matrices, respectively, and are assumed to be real and symmetric. The reconstructed structural vibration displacement signal is input into the second-order blind source separation algorithm to obtain the modal vibration matrix A. For a small damping structure, the reconstructed structural vibration displacement signal is expressed in modal coordinates as: Where A is the s-order mode shape vector A h The modal vibration matrix q(t) is composed of the modal response signals q h (t) composed of non-uniform modal coordinate independent samples, h = 1, 2, ..., s, q (t) = [q1 (t), q2 (t), ..., q s (t)] T ; The non-uniform modal coordinate independent sample q(t) is calculated by the above formula; The modal vibration matrix A is judged for positive and negative signs and regularized to obtain the corresponding structural vibration shape.

5. The structural modal recognition method based on under-sampling video according to claim 4 is characterized in that: The non-uniform modal coordinate independent samples are processed by a compressed sensing algorithm to restore the displacement signal in a uniform state, specifically including: The following formula is used to process the non-uniform modal coordinate independent samples: ω=arg min||ω||1,stH(t)=AΨω; Where A is the mode shape matrix, is the discrete Fourier basis, q(t) is the independent sample of the non-uniform modal coordinates, H(t) represents the displacement signal under the uniform state, ω=e -2πk / N is the Nth primitive root of unity, k 2 =-1, 6. The structural modal recognition method based on under-sampling video according to claim 3 is characterized in that: The vibration video is acquired by using a camera, and the sampling rate of the camera is f s The range is 30-60Hz.

7. A structural modal recognition device based on under-sampled video, characterized in that: include: A signal acquisition module is configured to acquire a vibration video of the structure, and obtain a velocity signal and a displacement signal of the structural vibration by using an LK optical flow method and a template matching method based on the vibration video; A data reconstruction module is configured to perform interpolation based on the velocity signal and the displacement signal to obtain a reconstructed structural vibration displacement signal; A signal processing module is configured to process the reconstructed structural vibration displacement signal using a blind source separation algorithm to obtain independent samples of structural vibration shapes and non-uniform modal coordinates; The frequency calculation module is configured to process the non-uniform modal coordinate independent samples through a compressed sensing algorithm to restore the displacement signal in a uniform state, and use fast Fourier transform on the displacement signal in the uniform state to obtain the frequency of structural vibration.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images