A method, system, terminal and storage medium for monitoring full-field vibration of structure based on long- and short-focus cameras

Abstract

The present invention relates to the technical field of structural health monitoring, and discloses a method, a system, a terminal and a storage medium for structural full-field vibration monitoring based on long-focus and short-focus cameras. The method includes: acquiring a local image captured by a long-focus camera and obtaining a main vibration frequency based on the local image; acquiring a global video captured by a short-focus camera, and performing spatial decomposition on the video frame image signal corresponding to the global video by using a complex-valued pyramid transform to obtain subband signals; setting parameters of a band-pass filter according to the main vibration frequency, performing band-pass filtering and amplification on the phase time series corresponding to the subband signals, then performing phase modulation on the subband signals to obtain new subband signals, and reconstructing the new subband signals through video to obtain a target amplified video; and obtaining a vibration displacement signal based on the target amplified video. The present invention realizes non-contact, large-range and high-resolution structural vibration monitoring through the fusion of long-focus and short-focus cameras and the phase-based video motion amplification technology.

Description

Technical Field

[0001] The present invention relates to the technical field of structural health monitoring, and particularly to a method, system, terminal and computer-readable storage medium for structural full-field vibration monitoring based on long and short focal length cameras. Background Art

[0002] With the development of Structural Health Monitoring (SHM), non-contact and high-precision measurement methods have become key requirements in many engineering and scientific applications. Traditional contact monitoring devices, such as accelerometers, vibration sensors, laser rangefinders, etc., although having high precision in many applications, still have several limitations. These traditional devices usually need to be in contact or close contact with the object to be measured, resulting in complex installation, additional maintenance costs, and possible interference with the object to be measured. In addition, in some scenarios requiring non-contact, long-term monitoring or large-scale monitoring, traditional devices cannot provide flexible and efficient solutions.

[0003] Moreover, existing structural vibration monitoring technologies usually need to make a trade-off between large range and high resolution, and cannot have the measurement capabilities of both large range and high resolution at the same time. For example, the optical flow method is good at large-scale monitoring but has insufficient resolution; the structured light method can provide high local resolution but cannot achieve large-scale monitoring. In practical applications, for structural vibration monitoring and health assessment technologies, it is often required to have the measurement capabilities of both large range and high resolution at the same time.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the Invention

[0005] The main purpose of the present invention is to provide a method, system, terminal and computer-readable storage medium for structural full-field vibration monitoring based on long and short focal length cameras, aiming to solve the problems of poor non-contact performance and inability to balance large-scale monitoring and high-resolution measurement in existing structural vibration monitoring technologies.

[0006] To achieve the above invention purpose, the present invention provides a method for structural full-field vibration monitoring based on long and short focal length cameras, and the method for structural full-field vibration monitoring based on long and short focal length cameras includes:

[0007] Obtain a local image obtained by a long focal length camera taking a local shot of a key part of a target structure, and process the local image to obtain the main vibration frequency of the target structure;

[0008] Obtain a global video obtained by a short focal length camera taking a global shot of the target structure, and perform spatial decomposition processing on the video frame image signal corresponding to the global video by using a complex-valued pyramid transform to obtain sub-band signals of different scales and different directions;

[0009] Set the parameters of the band - pass filter according to the main vibration frequency, and perform band - pass filtering on the phase time series corresponding to the sub - band signal using the band - pass filter to obtain a target phase time series, and perform amplification processing on the target phase time series to obtain a new phase time series;

[0010] Perform phase modulation on the sub - band signal according to the new phase time series to obtain a new sub - band signal, and perform video reconstruction according to the new sub - band signals of different scales and different directions to obtain a target amplified video;

[0011] Perform landmark locking and vibration tracking on the target amplified video to obtain the vibration displacement signal of the landmark in the target amplified video.

[0012] Optionally, the obtaining of the local image obtained by the long - focal camera for local shooting of the key part of the target structure and the processing of the local image to obtain the main vibration frequency of the target structure specifically includes:

[0013] Obtain the local image obtained by the long - focal camera for local shooting of the key part of the target structure with landmarks pasted thereon;

[0014] Lock the landmark in the local image by using the corner detection method or the template matching method;

[0015] Track the spatial position change of the landmark vibration to obtain an initial vibration displacement signal;

[0016] Perform a fast Fourier transform on the initial vibration displacement signal to obtain the frequency spectrum curve corresponding to the initial vibration displacement signal, and use the frequency corresponding to the point with the largest amplitude in the frequency spectrum curve as the main vibration frequency of the target structure.

[0017] Optionally, the obtaining of the global video obtained by the short - focal camera for global shooting of the target structure and the spatial decomposition processing of the video - frame image signal corresponding to the global video using the complex - valued pyramid transform to obtain sub - band signals of different scales and different directions specifically includes:

[0018] Obtain the global video obtained by the short - focal camera for global shooting of the target structure, and extract video frames from the global video to obtain a video - frame image sequence corresponding to the global video;

[0019] Construct a time - domain signal that changes with time according to the intensity change of each pixel in the video - frame image sequence and in accordance with the order of the video - frame images in the video - frame image sequence;

[0020] Perform a Fourier transform on the time - domain signal to obtain the video - frame image signal corresponding to the global video:

[0021] ;

[0022] Among them, represents the video frame image signal, represents the horizontal axis component of the pixel coordinates, represents the vertical axis component of the pixel coordinates, represents time, represents the horizontal axis component of the spatial frequency, represents the vertical axis component of the spatial frequency, represents the frequency component, represents the amplitude response of the video frame image signal, represents the phase change of the video frame image signal, represents the imaginary unit;

[0023] Perform multi-scale spatial decomposition processing on the video frame image signal using a complex-valued pyramid transform to obtain sub-band signals of different scales and different directions:

[0024] ;

[0025] Among them, represents the sub-band signal corresponding to the frequency component , represents the amplitude information of the sub-band signal, represents the phase time series corresponding to the sub-band signal, represents the pixel displacement.

[0026] Optionally, set the parameters of the band-pass filter according to the main vibration frequency, and perform band-pass filtering on the phase time series corresponding to the sub-band signal using the band-pass filter to obtain a target phase time series, and perform amplification processing on the target phase time series to obtain a new phase time series, specifically including:

[0027] Set the parameters of the band-pass filter according to the main vibration frequency, set the main vibration frequency as the center frequency of the band-pass filter, and set the set-length frequency band centered on the main vibration frequency as the bandwidth of the band-pass filter;

[0028] Perform band-pass filtering on the phase time series corresponding to the sub-band signal using the band-pass filter with the set parameters to obtain a target phase time series:

[0029] ;

[0030] Among them, represents the target phase time series corresponding to the frequency component ;

[0031] Perform amplification processing on the target phase time series to obtain a new phase time series:

[0032] ;

[0033] wherein, represents the new phase time series corresponding to the frequency component , and represents the amplification factor.

[0034] Optionally, phase-modulating the sub-band signal according to the new phase time series to obtain a new sub-band signal, and performing video reconstruction according to the new sub-band signals of different scales and different directions to obtain a target amplified video, specifically including:

[0035] Phase-modulating the sub-band signal according to the new phase time series to obtain a new sub-band signal:

[0036] ;

[0037] wherein, represents the new sub-band signal corresponding to the frequency component ;

[0038] Performing video reconstruction on the new sub-band signals of different scales and different directions to obtain a target amplified video:

[0039] ;

[0040] wherein, represents the amplified video frame image signal corresponding to the target amplified video.

[0041] Optionally, performing landmark locking and vibration tracking on the target amplified video to obtain the vibration displacement signal of the landmark in the target amplified video, specifically including:

[0042] Using the corner detection method or the template matching method to lock the landmark in the amplified video frame image corresponding to the target amplified video;

[0043] Tracking the spatial position change of the landmark vibration to obtain the vibration displacement signal;

[0044] Obtaining the structural full-field vibration monitoring result according to the vibration displacement signal;

[0045] wherein, the amplified video frame image corresponds to the amplified video frame image signal.

[0046] Optionally, before using the corner detection method or the template matching method to lock the landmark in the amplified video frame image corresponding to the target amplified video, it further includes:

[0047] According to the key part, an interested area including the key part is intercepted from the magnified video frame images corresponding to the target magnified video.

[0048] To achieve the above invention purpose, the present invention also provides a structural full-field vibration monitoring system based on a long-focus and a short-focus camera. The structural full-field vibration monitoring system based on a long-focus and a short-focus camera includes:

[0049] Main vibration frequency acquisition module: used to acquire the local images obtained by the long-focus camera for local shooting of the key part of the target structure, and process the local images to obtain the main vibration frequency of the target structure;

[0050] Sub-band signal acquisition module: used to acquire the global video obtained by the short-focus camera for global shooting of the target structure, and perform spatial decomposition processing on the video frame image signals corresponding to the global video by using complex-valued pyramid transform to obtain sub-band signals of different scales and different directions;

[0051] New phase time series acquisition module: used to set the parameters of the band-pass filter according to the main vibration frequency, and perform band-pass filtering processing on the phase time series corresponding to the sub-band signals by using the band-pass filter to obtain the target phase time series, and perform magnification processing on the target phase time series to obtain a new phase time series;

[0052] Target magnified video acquisition module: used to perform phase modulation on the sub-band signals according to the new phase time series to obtain new sub-band signals, and perform video reconstruction according to the new sub-band signals of different scales and different directions to obtain the target magnified video;

[0053] Vibration displacement signal acquisition module: used to perform landmark locking and vibration tracking on the target magnified video to obtain the vibration displacement signal of the landmark in the target magnified video.

[0054] To achieve the above invention purpose, the present invention also provides a terminal. The terminal includes: a memory, a processor, and a structural full-field vibration monitoring program based on a long-focus and a short-focus camera stored on the memory and executable on the processor. When the structural full-field vibration monitoring program based on a long-focus and a short-focus camera is executed by the processor, the steps of the above-mentioned structural full-field vibration monitoring method based on a long-focus and a short-focus camera are implemented.

[0055] To achieve the above invention purpose, the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a structural full-field vibration monitoring program based on a long-focus and a short-focus camera. When the structural full-field vibration monitoring program based on a long-focus and a short-focus camera is executed by a processor, the steps of the above-mentioned structural full-field vibration monitoring method based on a long-focus and a short-focus camera are implemented.

[0056] In the present invention, a local image obtained by a long - focal - length camera taking a local shot of a key part of a target structure is acquired, and the local image is processed to obtain the main vibration frequency of the target structure; a global video obtained by a short - focal - length camera taking a global shot of the target structure is acquired, and a complex - valued pyramid transform is used to perform spatial decomposition processing on the video - frame image signal corresponding to the global video to obtain sub - band signals of different scales and different directions; parameters of a band - pass filter are set according to the main vibration frequency, and the band - pass filter is used to perform band - pass filtering on the phase time series corresponding to the sub - band signals to obtain a target phase time series, and the target phase time series is amplified to obtain a new phase time series; the sub - band signals are phase - modulated according to the new phase time series to obtain new sub - band signals, and video reconstruction is performed according to the new sub - band signals of different scales and different directions to obtain a target amplified video; landmark locking and vibration tracking are performed on the target amplified video to obtain the vibration displacement signal of the landmark in the target amplified video. Through the phase - based video motion amplification technology, the present invention can achieve high - precision micro - displacement measurement without contact; by combining the single - point high - resolution data provided by the long - focal - length camera and the large - range field of view provided by the short - focal - length camera, it can not only monitor large - range vibrations but also accurately measure local micro - displacements; using standard industrial camera equipment, non - contact measurement is realized, reducing the cost of the system and simplifying the installation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flowchart of a preferred embodiment of the method for monitoring the full - field vibration of a structure based on long - and short - focal - length cameras of the present invention;

[0058] Figure 2 is a schematic diagram of a long - focal - length camera and a short - focal - length camera of the present invention taking pictures of a target structure;

[0059] Figure 3 is a structural diagram of a preferred embodiment of the system for monitoring the full - field vibration of a structure based on long - and short - focal - length cameras of the present invention;

[0060] Figure 4 is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0062] With the development of structural health monitoring (SHM), non-contact, high-precision measurement methods have become a key requirement in many engineering and scientific applications. Traditional contact monitoring equipment, such as accelerometers, vibration sensors, laser rangefinders, etc., have high accuracy in many applications, but still have several limitations. These traditional devices usually require contact or close contact with the object being measured, resulting in complex installation, additional maintenance costs, and possible interference with the object being measured. In addition, in some scenarios that require non-contact, long-term monitoring or large-scale monitoring, traditional equipment cannot provide flexible and efficient solutions.

[0063] As an emerging non-contact measurement method, video monitoring technology is increasingly used in the field of structural health monitoring. Video monitoring technology captures tiny movements on the surface of an object through collected video images, and uses image processing and analysis technology to achieve accurate measurement of motion trajectory, displacement and vibration. In particular, in vibration monitoring, video technology not only avoids the potential impact of physical contact on objects, but also provides a wider range of application scenarios through computer vision processing. However, existing video monitoring technology usually needs to make a trade-off between large range and high resolution, and cannot have the measurement capabilities of large range and high resolution at the same time. For example, the optical flow method is good at large-scale monitoring, but the resolution is insufficient; the structured light method can provide higher local resolution, but cannot achieve large-scale monitoring. In practical applications, for structural vibration monitoring and health assessment technology, it is often required to have the measurement capabilities of large range and high resolution at the same time.

[0064] Existing structural vibration monitoring technologies can be divided into the following categories: 1) Accelerometer and vibration sensor technology: Accelerometers are currently widely used in vibration monitoring and structural health monitoring. Accelerometers measure the acceleration changes of objects and then infer displacement or vibration. 2) Laser rangefinders and laser scanning technology: The displacement of an object is calculated by irradiating the surface of an object with a laser beam and measuring the time difference of the reflected light. Laser technology usually has high measurement accuracy. 3) Computer vision and video motion analysis: Computer vision technology, especially video-based motion analysis, has gradually become a promising non-contact measurement technology. Existing video-based motion analysis methods are mainly carried out through the following technical means: 31) Optical flow method: Estimate the motion and displacement of an object by analyzing the pixel changes between video frames; 32) Structured light method and marking method: The displacement of an object is calculated by projecting known light stripes or marks on the surface of the object and tracking the displacement of these light spots.

[0065] The above-mentioned existing technologies have the following disadvantages: 1) Limitations and resolution issues: Although existing video-based motion analysis methods (such as optical flow) can monitor vibrations or displacements over a large range, their accuracy is insufficient, especially for the measurement accuracy of small displacements. For subtle movements or high-frequency vibrations, existing technologies are difficult to provide sufficient accuracy and reliability. 2) Equipment installation and environmental interference: Although devices such as laser rangefinders and accelerometers can provide high-precision data, their installation is complicated and susceptible to environmental interference. For example, accelerometers must be in direct contact with the object being measured, which may affect the measurement results, especially in flexible structures or structures with large dynamic changes. 3) High requirements for lighting and texture: Many video-based monitoring methods rely on good lighting and texture features on the surface of objects, which imposes great limitations on certain environmental conditions (such as low light, reflections or smooth surfaces), resulting in unstable measurement results or reduced accuracy. 4) Unable to take into account both large range and high resolution: Existing technologies usually choose to make a trade-off between large range and high resolution. Some technologies are good at large-scale monitoring (such as optical flow), while others provide high local resolution (such as structured light). However, in practical applications, both large-scale and high-resolution measurement capabilities are often required. 5) High cost and complexity: Existing laser rangefinders, accelerometers, and complex computer vision algorithms usually require expensive equipment and complex post-processing, which increases the overall cost and usage threshold of the system.

[0066] To solve the above technical problems, the present invention provides a method for monitoring the full-field vibration of a structure based on long and short focal length cameras. It acquires local images obtained by the long focal length camera taking local pictures of the key parts of the target structure, and processes the local images to obtain the main vibration frequency of the target structure. It acquires the global video obtained by the short focal length camera taking global pictures of the target structure, and performs spatial decomposition processing on the video frame image signals corresponding to the global video by using the complex-valued pyramid transform to obtain sub-band signals of different scales and different directions. It sets the parameters of the band-pass filter according to the main vibration frequency, and uses the band-pass filter to perform band-pass filtering on the phase time series corresponding to the sub-band signals to obtain the target phase time series, and performs amplification processing on the target phase time series to obtain a new phase time series. It performs phase modulation on the sub-band signals according to the new phase time series to obtain new sub-band signals, and performs video reconstruction according to the new sub-band signals of different scales and different directions to obtain the target amplified video. It performs marker point locking and vibration tracking on the target amplified video to obtain the vibration displacement signal of the marker point in the target amplified video. Through the phase-based video motion amplification technology, the present invention can achieve high-precision micro-displacement measurement without contact; by combining the single-point high-resolution data provided by the long focal length camera and the large-range field of view provided by the short focal length camera, it can not only monitor large-range vibrations but also accurately measure local micro-displacements; using standard industrial camera equipment, compared with traditional laser rangefinders and accelerometers, it realizes non-contact measurement, reduces the cost of the system, and simplifies the installation process.

[0067] The following further describes the application content by describing the embodiments in conjunction with the accompanying drawings.

[0068] A preferred embodiment of the method for monitoring the full-field vibration of a structure based on long and short focal length cameras according to the present invention is as Figure 1 shown, and specifically includes:

[0069] S1. Acquire local images obtained by the long focal length camera taking local pictures of the key parts of the target structure, and process the local images to obtain the main vibration frequency of the target structure.

[0070] In an implementation manner of this embodiment, the acquiring local images obtained by the long focal length camera taking local pictures of the key parts of the target structure, and processing the local images to obtain the main vibration frequency of the target structure specifically includes:

[0071] Acquire local images obtained by the long focal length camera taking local pictures of the key parts of the target structure where marker points are pasted;

[0072] Use the corner detection method or the template matching method to lock the marker points in the local images;

[0073] Track the change in the spatial position of the vibration of the marked points to obtain an initial vibration displacement signal;

[0074] Perform a fast Fourier transform on the initial vibration displacement signal to obtain the frequency spectrum curve corresponding to the initial vibration displacement signal, and use the frequency corresponding to the point with the largest amplitude in the frequency spectrum curve as the main vibration frequency of the target structure.

[0075] Specifically, the present invention combines the characteristics of a long - focal - length camera and a short - focal - length camera, makes full use of the high - resolution images obtained by the long - focal - length camera in a local area, extracts the fundamental frequency information of the structure as the basis for filter parameters, and then performs phase - based video processing on the wide - field video stream captured by the short - focal - length camera. The structural fundamental frequency data is obtained by the long - focal - length camera taking local pictures of the key parts of the target structure (such as beam ends, supports, nodes, etc.). If the shooting position has good texture features, it can be directly shot; otherwise, artificial marked points can be pasted, so that the image is stable and has a high contrast, which is beneficial to subsequent tracking and analysis. The process of obtaining the structural fundamental frequency data includes: using corner detection (such as Harris corner detection) or template matching to lock the marked points, tracking the spatial position of the vibration of the marked points to obtain a time - series signal (i.e., the initial vibration displacement signal), using the fast Fourier transform (FFT, Fast Fourier Transform) to analyze and obtain the frequency spectrum characteristics (i.e., the frequency spectrum curve) of the time - series signal of the vibration of the marked points during the measurement period, and clearly extracting the main vibration frequency of the structure (i.e., the structural fundamental frequency data) from the frequency spectrum curve, which is used as an important basis for selecting the video motion amplification frequency band, that is, selecting the filter bandwidth for processing the short - focal - length camera video stream based on the structural fundamental frequency data extracted by the long - focal - length camera. As Figure 2 As shown, the present invention uses the long - focal - length camera 1 to take local pictures of the key parts of the target structure 3 with marked points to obtain local images (i.e., high - resolution images), and uses the short - focal - length camera 2 to take global pictures of the target structure 3 to obtain global videos (i.e., large - range videos / wide - field videos). Among them, the long - focal - length camera 1 and the short - focal - length camera 2 work synchronously, and data synchronization acquisition is ensured through timing control.

[0076] S2. Obtain the global video obtained by the short - focal - length camera taking a global picture of the target structure, and perform spatial decomposition processing on the video frame image signal corresponding to the global video using a complex - valued pyramid transform to obtain sub - band signals of different scales and different directions.

[0077] In an implementation manner of this embodiment, the step of obtaining the global video obtained by the short - focal - length camera taking a global picture of the target structure, and performing spatial decomposition processing on the video frame image signal corresponding to the global video using a complex - valued pyramid transform to obtain sub - band signals of different scales and different directions specifically includes:

[0078] Obtain the global video captured by the short - focus camera for the target structure, and extract video frames from the global video to obtain the video frame image sequence corresponding to the global video;

[0079] According to the intensity change of each pixel in the video frame image sequence, and in the order of the video frame images in the video frame image sequence, construct a time - varying time - domain signal;

[0080] Perform a Fourier transform on the time - domain signal to obtain the video frame image signal corresponding to the global video. That is, first, perform frequency - domain analysis on the image signal (referring to the time - domain signal) through Fourier transform. The intensity of a two - dimensional image can be expressed as the superposition of frequency components by Fourier series:

[0081] ;

[0082] Among them, represents the video frame image signal, represents the horizontal axis component of the pixel coordinate , represents the pixel coordinate 's vertical axis component, represents time, represents the horizontal axis component of the spatial frequency , represents the spatial frequency 's vertical axis component, represents the frequency component, represents the amplitude response of the video frame image signal, represents the phase change of the video frame image signal, represents the imaginary unit;

[0083] Perform multi - scale spatial decomposition processing on the video frame image signal using complex - valued pyramid transform to obtain sub - band signals of different scales and different directions. This pyramid structure is a multi - scale, multi - direction band - pass filter bank that can represent both the amplitude information and phase information of an image (referring to the sub - band signal. It should be noted that the sub - band signal in the present invention can be understood as a sub - band image, and the video frame image signal can also be understood as a video frame image) in complex form. The expression of the sub - band signal for each frequency component is:

[0084] ;

[0085] Among them, represents the sub - band signal corresponding to the frequency component ; represents the amplitude information of the sub - band signal corresponding to the frequency component , indicating the intensity of the image pixel at the frequency component ; Represents the phase time series corresponding to the sub-band signal; Represents the pixel displacement caused by image motion (i.e., the motion of the landmark points in the image) that changes over time.

[0086] Specifically, the core idea of the phase motion amplification method is to regard the intensity change of each pixel in the image sequence as a time-domain signal that changes over time, use Fourier analysis technology to convert the motion signal of the pixel (referring to the above time-domain signal, i.e., the pixel intensity change signal) to the frequency domain, and realize the amplification of the motion signal by analyzing the phase information of the frequency-domain signal to effectively enhance the tiny motion signal. It should be noted that when an object undergoes tiny motion (such as vibration, deformation), its projection position in the image will change over time, resulting in the change of the brightness (intensity) at the same pixel position due to object movement or light change. Since it is difficult to directly measure sub-pixel motion, the change of intensity over time is regarded as an indirect representation of motion, and the intensity change signal implies displacement information.

[0087] S3. Set the parameters of the band-pass filter according to the main vibration frequency, and perform band-pass filtering on the phase time series corresponding to the sub-band signal by using the band-pass filter to obtain a target phase time series, and perform amplification processing on the target phase time series to obtain a new phase time series.

[0088] In an implementation manner of this embodiment, the setting the parameters of the band-pass filter according to the main vibration frequency, and performing band-pass filtering on the phase time series corresponding to the sub-band signal by using the band-pass filter to obtain a target phase time series, and performing amplification processing on the target phase time series to obtain a new phase time series specifically includes:

[0089] Set the parameters of the band-pass filter according to the main vibration frequency, set the main vibration frequency as the center frequency of the band-pass filter, and set the frequency band with a set length centered on the main vibration frequency as the bandwidth of the band-pass filter;

[0090] Perform band-pass filtering on the phase time series corresponding to the sub-band signal by using the band-pass filter with the set parameters to obtain a target phase time series:

[0091] ;

[0092] wherein, Represents the frequency component The corresponding target phase time series;

[0093] Perform amplification processing on the target phase time series, that is, multiply the filtered phase by the amplification factor (i.e., the amplification coefficient) , to obtain a new phase time series:

[0094] ;

[0095] Among them, represents the new phase time series corresponding to the frequency component ; represents the amplification factor which is greater than 0 and is used to enhance the motion.

[0096] Specifically, the core of the present invention lies in extracting minute motion information from the local phase change of the image. According to the Fourier shift theorem, if the image undergoes translation, its phase in the complex domain representation will change linearly accordingly. Therefore, the phase of the sub-band signal contains displacement information. To highlight the motion components at specific frequencies and simultaneously suppress low-frequency drift and high-frequency noise, band-pass filtering is performed on the phase time series of each point (i.e., pixel). Among them, when performing phase filtering on the global video of the short-focus camera, the parameters of the time-domain band-pass filter are set as follows: the center frequency of the short-focus camera (global video processing), ; the bandwidth of the short-focus camera (global video processing) is selected. Generally speaking, a narrow frequency band near the main peak frequency (i.e., the main vibration frequency ) is used as the amplification frequency band to ensure that the focus of motion amplification is concentrated in the region of the true dynamic response of the structure and to avoid the amplification of noise or other irrelevant frequency components. The typical value is .

[0097] S4. Perform phase modulation on the sub-band signal according to the new phase time series to obtain a new sub-band signal, and perform video reconstruction based on the new sub-band signals of different scales and different directions to obtain the target amplified video.

[0098] In an implementation manner of this embodiment, the performing phase modulation on the sub-band signal according to the new phase time series to obtain a new sub-band signal, and performing video reconstruction based on the new sub-band signals of different scales and different directions to obtain the target amplified video specifically includes:

[0099] Perform phase modulation on the sub-band signal according to the new phase time series to obtain a new sub-band signal:

[0100] ;

[0101] Among them, represents the new sub-band signal corresponding to the frequency component ;

[0102] Perform video reconstruction on the new sub-band signals of different scales and different directions to obtain the target amplified video:

[0103] ;

[0104] Wherein, represents the magnified video frame image signal corresponding to the target magnified video.

[0105] Specifically, after performing phase adjustment on sub-bands of all scales and directions, the image frames at each moment (i.e., magnified video frame images) are re-synthesized, and the entire video sequence (i.e., the target magnified video) is reconstructed in reverse. This step obtains the final motion magnified video by synthesizing the sub-band signals of each frequency component. Through the phase changes in the video sequence, extremely tiny displacement information can be extracted. Combining with the video motion magnification technology, these tiny displacements become obvious, thus providing high-precision measurement data without contacting the object. Compared with the accuracy of traditional accelerometers or laser rangefinders, the video motion magnification technology can provide more detailed and efficient displacement monitoring.

[0106] S5. Perform fiducial point locking and vibration tracking on the target magnified video to obtain the vibration displacement signal of the fiducial point in the target magnified video.

[0107] In an implementation manner of this embodiment, the performing fiducial point locking and vibration tracking on the target magnified video to obtain the vibration displacement signal of the fiducial point in the target magnified video specifically includes:

[0108] Adopt the corner detection method or the template matching method to lock the fiducial point in the magnified video frame image corresponding to the target magnified video;

[0109] Track the spatial position change of the fiducial point vibration to obtain the vibration displacement signal;

[0110] Obtain the structural full-field vibration monitoring result according to the vibration displacement signal;

[0111] Wherein, the magnified video frame image corresponds to the magnified video frame image signal.

[0112] In an implementation manner of this embodiment, before adopting the corner detection method or the template matching method to lock the fiducial point in the magnified video frame image corresponding to the target magnified video, it further includes:

[0113] Intercept the region of interest containing the key part from the magnified video frame image corresponding to the target magnified video according to the key part.

[0114] Specifically, first, perform ROI (Region of Interest) selection on the processed motion magnification video (i.e., the target magnification video). According to the key parts (i.e., the key vibration response regions selected according to the design drawing or experience), intercept the ROI region from the magnified video frame images corresponding to the target magnification video; then use corner detection (such as Harris corner detection) or template matching to lock the landmark points in the ROI region, track the spatial positions of the vibrations of the landmark points, obtain a new time series signal (i.e., the vibration displacement signal), and further obtain the vibration information at the positions where the landmark points are pasted on the target structure (i.e., the key parts), so that the tiny vibration information of the structure that is difficult to detect in the large field of view of the original short-focus camera can be accurately extracted. It should be noted that in step S4, the sub-bands of different frequency components can also be directly synthesized into magnified video frame images, and in step S5, image processing is directly performed on the magnified video frame images to obtain the vibration displacement signal, that is, skipping the video synthesis step.

[0115] The present invention aims at the problems of poor non-contact performance, limited measurement range, insufficient resolution, etc. existing in the current structural vibration monitoring and health assessment technologies, and proposes a structural full-field vibration monitoring method based on long-focus and short-focus cameras. This method combines the characteristics of long-focus and short-focus cameras, makes full use of the high-resolution images obtained by the long-focus camera in the local area, extracts the fundamental frequency information of the structure as the basis for filter parameters, and then performs phase-based video processing on the wide-field video stream captured by the short-focus camera. Select the filter bandwidth for processing the data stream of the short-focus camera according to the fundamental frequency information of the structure extracted by the long-focus camera, analyze the local phase change information in the image, accurately extract the tiny displacement signal of the structure in a specific frequency band, effectively separate the dynamic response of the target structure from the background interference, and realize the magnification of tiny vibrations and displacement extraction. The present invention uses the long-focus camera to provide single-point high-resolution data and the short-focus camera to provide a large field of view. Through the fusion of the long-focus and short-focus cameras, large-range monitoring and high-precision local measurement are simultaneously achieved, avoiding the trade-off between these two aspects in the prior art. Image data is remotely collected by camera equipment without contacting the object to be measured, avoiding the interference brought by traditional sensors and realizing true non-contact measurement. Non-contact measurement ensures that the true response of the structure is not affected by external interference and can monitor any irregularly shaped or sensitive structure for a long time under dynamic conditions. For large-scale structures (such as bridges, buildings, etc.) or complex environments (such as bad weather, dangerous areas), this technology provides greater flexibility and operability.

[0116] Compared with traditional contact or local monitoring technologies such as accelerometers and laser rangefinders, the present invention does not need to install traditional contact sensors (such as accelerometers and laser rangefinders) on the surface of the structure, and can achieve long-distance, non-contact, and high-precision micro-vibration monitoring of large structures under actual working conditions. Through the fusion of long- and short-focus cameras and phase-based video motion amplification technology, the present invention can comprehensively monitor large-scale structures under various environmental conditions, while providing high-resolution detail data, significantly improving the detection accuracy of small displacements.

[0117] The technical advantages of the present invention are mainly reflected in the non-contact detection method, simple deployment, strong adaptability, and applicability to a variety of complex environments and large-scale structural scenes, and can achieve true non-contact measurement; by integrating video information of different focal lengths, it breaks through the traditional trade-off limitations between large-scale monitoring and high-resolution measurement, while ensuring the monitoring coverage, it improves the accuracy and timeliness of local measurements; through phase-based video motion amplification technology, it can effectively amplify tiny displacements and improve detection accuracy.

[0118] In addition, based on the above-mentioned structural full-field vibration monitoring method based on long-short focal length cameras, the present invention also provides a structural full-field vibration monitoring system based on long-short focal length cameras, wherein a preferred embodiment of the structural full-field vibration monitoring system based on long-short focal length cameras is as follows: Figure 3 As shown, specifically including:

[0119] Main vibration frequency acquisition module 01: used to acquire a local image obtained by partially photographing a key part of a target structure with a telephoto camera, and process the local image to obtain the main vibration frequency of the target structure;

[0120] Subband signal acquisition module 02: used to acquire a global video obtained by globally photographing the target structure with a short-focus camera, and perform spatial decomposition processing on the video frame image signal corresponding to the global video using a complex-valued pyramid transform to obtain subband signals of different scales and directions;

[0121] New phase time series acquisition module 03: used to set the parameters of the bandpass filter according to the main vibration frequency, and use the bandpass filter to perform bandpass filtering on the phase time series corresponding to the sub-band signal to obtain a target phase time series, and amplify the target phase time series to obtain a new phase time series;

[0122] The target magnified video acquisition module 04 is used to perform phase modulation on the sub-band signal according to the new phase time sequence to obtain a new sub-band signal, and perform video reconstruction according to the new sub-band signals of different scales and directions to obtain a target magnified video;

[0123] Vibration displacement signal acquisition module 05: It is used to lock the landmark points and track the vibration of the target magnified video, and obtain the vibration displacement signal of the landmark points in the target magnified video.

[0124] In addition, based on the above-mentioned structural full-field vibration monitoring method and system based on long and short focal length cameras, the present invention also correspondingly provides a terminal. Among them, in a preferred embodiment of the terminal, as Figure 4 shown, it specifically includes a processor 10, a memory 20, and a display 30. Figure 4 Only some components of the terminal are shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0125] The memory 20 can be an internal storage unit of the terminal in some embodiments, such as the hard disk or memory of the terminal. The memory 20 can also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, and a Flash Card equipped on the terminal, etc. Further, the memory 20 can also include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software installed on the terminal and various types of data, such as storing the program code of the terminal, etc. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, a structural full-field vibration monitoring program 40 based on long and short focal length cameras is stored on the memory 20, and the structural full-field vibration monitoring program 40 based on long and short focal length cameras can be executed by the processor 10, thereby implementing the steps of the structural full-field vibration monitoring method based on long and short focal length cameras in this application.

[0126] The processor 10 can be a Central Processing Unit (CPU), a microprocessor, or other data processing chips in some embodiments, and is used to run the program code stored in the memory 20 or process data, such as executing the structural full-field vibration monitoring program 40 based on long and short focal length cameras, etc.

[0127] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) toucher, etc. in some embodiments. The display 30 is used to display information on the terminal and to display a visual user interface.

[0128] In one embodiment, when the processor 10 executes the structural full-field vibration monitoring program 40 in the memory 20, the steps of the structural full-field vibration monitoring method based on long and short focal length cameras as described above are implemented.

[0129] The present invention also correspondingly provides a computer-readable storage medium. The computer-readable storage medium stores a structural full-field vibration monitoring program based on a long-short focus camera. When the structural full-field vibration monitoring program based on the long-short focus camera is executed by a processor, the steps of the structural full-field vibration monitoring method based on the long-short focus camera as described above are implemented.

[0130] It should be noted that in this text, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or terminal. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or terminal including that element.

[0131] Certainly, those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When the program is executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disc, etc.

[0132] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description. All such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A method for structural full-field vibration monitoring based on long- and short-focus cameras, characterized in that, The full-field vibration monitoring method based on long- and short-focus cameras includes: Obtaining local images captured by the long-focus camera of key parts of the target structure, and processing the local images to obtain the main vibration frequency of the target structure; Obtaining the global video captured by the short-focus camera of the target structure, and performing spatial decomposition processing on the video frame image signals corresponding to the global video using complex-valued pyramid transform to obtain sub-band signals of different scales and different directions, specifically including: Obtaining the global video captured by the short-focus camera of the target structure, and extracting video frames from the global video to obtain the video frame image sequence corresponding to the global video; Constructing a time-domain signal that changes with time according to the intensity change of each pixel in the video frame image sequence and in the order of the video frame images in the video frame image sequence; Performing Fourier transform on the time-domain signal to obtain the video frame image signal corresponding to the global video: ； Among them, represents the video frame image signal, represents the horizontal axis component of the pixel coordinates, represents the vertical axis component of the pixel coordinates, represents time, represents the horizontal axis component of the spatial frequency, represents the vertical axis component of the spatial frequency, represents the frequency component, represents the amplitude response of the video frame image signal, represents the phase change of the video frame image signal, represents the imaginary unit; Performing multi-scale spatial decomposition processing on the video frame image signal using complex-valued pyramid transform to obtain sub-band signals of different scales and different directions: ； Among them, represents the frequency component corresponding subband signal, represents the amplitude information of the subband signal, represents the phase time series corresponding to the subband signal, represents the pixel displacement; Setting the parameters of the band-pass filter according to the main vibration frequency, and using the band-pass filter to perform band-pass filtering on the phase time series corresponding to the sub-band signals to obtain the target phase time series, and performing amplification processing on the target phase time series to obtain a new phase time series, specifically including: Setting the parameters of the band-pass filter according to the main vibration frequency, setting the main vibration frequency as the center frequency of the band-pass filter, and setting the frequency band with a set length centered on the main vibration frequency as the bandwidth of the band-pass filter; Using the band-pass filter with the set parameters to perform band-pass filtering on the phase time series corresponding to the sub-band signals to obtain the target phase time series: ； Among them, represents the frequency component corresponding target phase time series; Performing amplification processing on the target phase time series to obtain a new phase time series: ； Among them, represents the frequency component corresponding new phase time series, represents the amplification factor; Performing phase modulation on the sub-band signals according to the new phase time series to obtain new sub-band signals, and performing video reconstruction according to the new sub-band signals of different scales and different directions to obtain the target amplified video; Performing landmark locking and vibration tracking on the target amplified video to obtain the vibration displacement signal of the landmark in the target amplified video.

2. The method for full-field vibration monitoring of the structure based on the long-short focus camera according to claim 1, wherein The obtaining of local images captured by the long-focus camera of key parts of the target structure, and processing the local images to obtain the main vibration frequency of the target structure specifically includes: Obtaining local images captured by the long-focus camera of the key parts of the target structure where landmarks are pasted; Locking the landmarks in the local images using the corner detection method or the template matching method; Tracking the spatial position change of the landmark vibration to obtain the initial vibration displacement signal; Performing fast Fourier transform on the initial vibration displacement signal to obtain the frequency spectrum curve corresponding to the initial vibration displacement signal, and taking the frequency corresponding to the point with the largest amplitude in the frequency spectrum curve as the main vibration frequency of the target structure.

3. The method for structural full-field vibration monitoring based on long- and short-focus cameras according to claim 1, characterized in that Performing phase modulation on the sub-band signals according to the new phase time series to obtain new sub-band signals, and performing video reconstruction based on the new sub-band signals of different scales and different directions to obtain a target magnified video, specifically including: Performing phase modulation on the sub-band signals according to the new phase time series to obtain new sub-band signals: ； Among them, represents the frequency component corresponding new sub-band signal; Performing video reconstruction on the new sub-band signals of different scales and different directions to obtain a target magnified video: ； Among them, represents the magnified video frame image signal corresponding to the target magnified video.

4. The method for structural full-field vibration monitoring based on long and short focal length cameras according to claim 3, wherein, Performing landmark locking and vibration tracking on the target magnified video to obtain the vibration displacement signal of the landmark in the target magnified video, specifically including: Locking the landmark in the magnified video frame image corresponding to the target magnified video by using the corner detection method or the template matching method; Tracking the spatial position change of the landmark vibration to obtain the vibration displacement signal; Obtaining the structural full-field vibration monitoring result according to the vibration displacement signal; Wherein, the magnified video frame image corresponds to the magnified video frame image signal.

5. The method for monitoring full-field vibration of a structure based on a long-short focal length camera according to claim 4, characterized in that: Before locking the landmark in the magnified video frame image corresponding to the target magnified video by using the corner detection method or the template matching method, it further includes: Intercepting the region of interest containing the key part from the magnified video frame image corresponding to the target magnified video according to the key part.

6. A structural full-field vibration monitoring system based on a long-short focus camera, characterized in that, The structural full-field vibration monitoring system based on long and short focal length cameras is applied to the structural full-field vibration monitoring method based on long and short focal length cameras according to any one of claims 1-5. The structural full-field vibration monitoring system based on long and short focal length cameras includes: Main vibration frequency acquisition module: used to acquire the local image obtained by the long focal length camera for local shooting of the key part of the target structure, and process the local image to obtain the main vibration frequency of the target structure; Sub-band signal acquisition module: used to acquire the global video obtained by the short focal length camera for global shooting of the target structure, and perform spatial decomposition processing on the video frame image signal corresponding to the global video by using the complex-valued pyramid transform to obtain sub-band signals of different scales and different directions; New phase time series acquisition module: used to set the parameters of the band-pass filter according to the main vibration frequency, and perform band-pass filtering processing on the phase time series corresponding to the sub-band signals by using the band-pass filter to obtain the target phase time series, and perform magnification processing on the target phase time series to obtain the new phase time series; Target magnified video acquisition module: used to perform phase modulation on the sub-band signals according to the new phase time series to obtain new sub-band signals, and perform video reconstruction based on the new sub-band signals of different scales and different directions to obtain the target magnified video; Vibration displacement signal acquisition module: used to perform landmark locking and vibration tracking on the target magnified video to obtain the vibration displacement signal of the landmark in the target magnified video.

7. A terminal, characterized in that, The terminal includes: a memory, a processor, and a structure full-field vibration monitoring program based on a long-short focus camera stored on the memory and operable on the processor. When the structure full-field vibration monitoring program based on the long-short focus camera is executed by the processor, the steps of the structure full-field vibration monitoring method based on the long-short focus camera as recited in any one of claims 1-5 are implemented.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a structure full-field vibration monitoring program based on a long-short focus camera. When the structure full-field vibration monitoring program based on the long-short focus camera is executed by a processor, the steps of the structure full-field vibration monitoring method based on the long-short focus camera as recited in any one of claims 1-5 are implemented.

Images