Structure full-field vibration monitoring method and system based on long and short-focus camera, terminal and storage medium

Abstract

The invention relates to the technical field of structural health monitoring, and discloses a long-focus and short-focus camera-based structural full-field vibration monitoring method and system, a terminal and a storage medium, and the method comprises the steps: obtaining a local image shot by a long-focus camera, and obtaining a main vibration frequency based on the local image; acquiring a global video shot by the short-focus camera, and performing spatial decomposition on a video frame image signal corresponding to the global video by adopting complex-valued pyramid transformation to obtain a sub-band signal; setting parameters of a band-pass filter according to the main vibration frequency, performing band-pass filtering and amplification on a phase time sequence corresponding to the sub-band signal, performing phase modulation on the sub-band signal to obtain a new sub-band signal, and performing video reconstruction on the new sub-band signal to obtain a target amplified video; and obtaining a vibration displacement signal based on the target amplified video. According to the invention, non-contact, large-range and high-resolution structural vibration monitoring is realized through fusion of the long-focus camera and the short-focus camera and a phase-based video motion amplification technology.

Description

Technical Field

[0001] The present invention relates to the technical field of structural health monitoring, and particularly relates to a method, a system, a terminal and a computer-readable storage medium for structural full-field vibration monitoring based on long and short focal length cameras. Background Art

[0002] With the development of Structural Health Monitoring (SHM), non-contact and high-precision measurement methods have become key requirements in many engineering and scientific applications. Traditional contact monitoring devices, such as accelerometers, vibration sensors, laser rangefinders, etc., although having high precision in many applications, still have several limitations. These traditional devices usually need to be in contact or in close contact with the object to be measured, resulting in complex installation, additional maintenance costs, and possible interference with the object to be measured. In addition, in some scenarios that require non-contact, long-term monitoring or large-scale monitoring, traditional devices cannot provide flexible and efficient solutions.

[0003] Moreover, existing structural vibration monitoring technologies usually need to make a trade-off between large range and high resolution, and cannot simultaneously have the measurement capabilities of large range and high resolution. For example, the optical flow method is good at large-scale monitoring but lacks resolution; the structured light method can provide high local resolution but cannot achieve large-scale monitoring. In practical applications, for structural vibration monitoring and health assessment technologies, it is often required to simultaneously have the measurement capabilities of large range and high resolution.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0005] The main purpose of the present invention is to provide a method, a system, a terminal and a computer-readable storage medium for structural full-field vibration monitoring based on long and short focal length cameras, aiming to solve the problems of poor non-contact performance and inability to balance large-scale monitoring and high-resolution measurement in existing structural vibration monitoring technologies.

[0006] To achieve the above-mentioned invention purpose, the present invention provides a method for structural full-field vibration monitoring based on long and short focal length cameras, and the method for structural full-field vibration monitoring based on long and short focal length cameras includes: Obtain a local image obtained by a long focal length camera taking a local shot of a key part of a target structure, and process the local image to obtain the main vibration frequency of the target structure; Obtain a global video obtained by a short focal length camera taking a global shot of the target structure, and perform spatial decomposition processing on the video frame image signal corresponding to the global video by using a complex-valued pyramid transform to obtain sub-band signals of different scales and different directions; Set the parameters of the band - pass filter according to the main vibration frequency, and perform band - pass filtering on the phase time series corresponding to the sub - band signal using the band - pass filter to obtain a target phase time series, and perform amplification processing on the target phase time series to obtain a new phase time series; Perform phase modulation on the sub - band signal according to the new phase time series to obtain a new sub - band signal, and perform video reconstruction based on the new sub - band signals of different scales and different directions to obtain a target amplified video; Perform landmark locking and vibration tracking on the target amplified video to obtain the vibration displacement signal of the landmark in the target amplified video.

[0007] Optionally, the obtaining of the local image obtained by the long - focal camera for local shooting of the key part of the target structure and the processing of the local image to obtain the main vibration frequency of the target structure specifically includes: Obtain the local image obtained by the long - focal camera for local shooting of the key part of the target structure where the landmark is pasted; Lock the landmark in the local image by using the corner detection method or the template matching method; Track the spatial position change of the landmark vibration to obtain an initial vibration displacement signal; Perform a fast Fourier transform on the initial vibration displacement signal to obtain the frequency spectrum curve corresponding to the initial vibration displacement signal, and take the frequency corresponding to the point with the largest amplitude in the frequency spectrum curve as the main vibration frequency of the target structure.

[0008] Optionally, the obtaining of the global video obtained by the short - focal camera for global shooting of the target structure and the spatial decomposition processing of the video - frame image signal corresponding to the global video using the complex - valued pyramid transform to obtain sub - band signals of different scales and different directions specifically includes: Obtain the global video obtained by the short - focal camera for global shooting of the target structure, and extract video frames from the global video to obtain a video - frame image sequence corresponding to the global video; Construct a time - varying time - domain signal according to the intensity change of each pixel in the video - frame image sequence and in accordance with the order of the video - frame images in the video - frame image sequence; Perform a Fourier transform on the time - domain signal to obtain the video - frame image signal corresponding to the global video: ; where, represents the video - frame image signal, represents the horizontal axis component of the pixel coordinate, represents the vertical axis component of the pixel coordinate, represents time, The horizontal axis component representing the spatial frequency, The vertical axis component representing the spatial frequency, The frequency component, The amplitude response representing the video frame image signal, The phase change representing the video frame image signal, The imaginary unit; Performing multi-scale spatial decomposition processing on the video frame image signal by using a complex-valued pyramid transform to obtain sub-band signals of different scales and different directions: ; Among them, The frequency component The corresponding sub-band signal, The amplitude information of the sub-band signal, The phase time series corresponding to the sub-band signal, The pixel displacement.

[0009] Optionally, setting the parameters of the band-pass filter according to the main vibration frequency, and performing band-pass filtering on the phase time series corresponding to the sub-band signal by using the band-pass filter to obtain a target phase time series, and performing amplification processing on the target phase time series to obtain a new phase time series, specifically including: Setting the parameters of the band-pass filter according to the main vibration frequency, setting the main vibration frequency as the center frequency of the band-pass filter, and setting the set-length frequency band centered on the main vibration frequency as the bandwidth of the band-pass filter; Performing band-pass filtering on the phase time series corresponding to the sub-band signal by using the band-pass filter with the set parameters to obtain a target phase time series: ; Among them, The frequency component The corresponding target phase time series; Performing amplification processing on the target phase time series to obtain a new phase time series: ; Among them, The frequency component The corresponding new phase time series, The amplification factor.

[0010] Optionally, performing phase modulation on the sub-band signal according to the new phase time series to obtain a new sub-band signal, and performing video reconstruction according to the new sub-band signals of different scales and different directions to obtain a target amplified video, specifically including: Phase-modulate the sub-band signals according to the new phase time series to obtain new sub-band signals: ; wherein, represents the new sub-band signal corresponding to the frequency component ; Reconstruct the new sub-band signals with different scales and different directions through video to obtain the target magnified video: ; wherein, represents the magnified video frame image signal corresponding to the target magnified video.

[0011] Optionally, the locking of the fiducial points and vibration tracking of the target magnified video to obtain the vibration displacement signal of the fiducial points in the target magnified video specifically includes: Lock the fiducial points in the magnified video frame image corresponding to the target magnified video by using the corner detection method or the template matching method; Track the spatial position change of the vibration of the fiducial points to obtain the vibration displacement signal; Obtain the structural full-field vibration monitoring result according to the vibration displacement signal; wherein, the magnified video frame image corresponds to the magnified video frame image signal.

[0012] Optionally, before locking the fiducial points in the magnified video frame image corresponding to the target magnified video by using the corner detection method or the template matching method, it further includes: Intercept the region of interest containing the key part from the magnified video frame image corresponding to the target magnified video according to the key part.

[0013] To achieve the above object of the invention, the present invention also provides a structural full-field vibration monitoring system based on long and short focal cameras, and the structural full-field vibration monitoring system based on long and short focal cameras includes: Main vibration frequency acquisition module: used to acquire the local image obtained by the long focal camera for local shooting of the key part of the target structure, and process the local image to obtain the main vibration frequency of the target structure; Sub-band signal acquisition module: used to acquire the global video obtained by the short focal camera for global shooting of the target structure, and perform spatial decomposition processing on the video frame image signal corresponding to the global video by using the complex-valued pyramid transform to obtain sub-band signals with different scales and different directions; New phase time series acquisition module: configured to set the parameters of a band-pass filter according to the main vibration frequency, perform band-pass filtering on the phase time series corresponding to the sub-band signal by using the band-pass filter to obtain a target phase time series, and perform amplification processing on the target phase time series to obtain a new phase time series; Target amplified video acquisition module: configured to perform phase modulation on the sub-band signal according to the new phase time series to obtain a new sub-band signal, and perform video reconstruction based on the new sub-band signals of different scales and different directions to obtain a target amplified video; Vibration displacement signal acquisition module: configured to perform landmark locking and vibration tracking on the target amplified video to obtain the vibration displacement signal of the landmark in the target amplified video.

[0014] To achieve the above invention purpose, the present invention also provides a terminal, the terminal includes: a memory, a processor, and a structure full-field vibration monitoring program based on a long and short focal length camera stored on the memory and executable on the processor. When the structure full-field vibration monitoring program based on the long and short focal length camera is executed by the processor, the steps of the above-mentioned structure full-field vibration monitoring method based on the long and short focal length camera are implemented.

[0015] To achieve the above invention purpose, the present invention also provides a computer-readable storage medium, the computer-readable storage medium stores a structure full-field vibration monitoring program based on a long and short focal length camera. When the structure full-field vibration monitoring program based on the long and short focal length camera is executed by a processor, the steps of the above-mentioned structure full-field vibration monitoring method based on the long and short focal length camera are implemented.

[0016] In the present invention, a local image obtained by a long - focal - length camera taking a local shot of a key part of a target structure is acquired, and the local image is processed to obtain the main vibration frequency of the target structure; a global video obtained by a short - focal - length camera taking a global shot of the target structure is acquired, and a complex - valued pyramid transform is used to perform spatial decomposition processing on the video - frame image signal corresponding to the global video to obtain sub - band signals of different scales and different directions; the parameters of a band - pass filter are set according to the main vibration frequency, and the band - pass filter is used to perform band - pass filtering on the phase time series corresponding to the sub - band signals to obtain a target phase time series, and the target phase time series is amplified to obtain a new phase time series; the sub - band signals are phase - modulated according to the new phase time series to obtain new sub - band signals, and video reconstruction is performed according to the new sub - band signals of different scales and different directions to obtain a target amplified video; landmark locking and vibration tracking are performed on the target amplified video to obtain the vibration displacement signal of the landmark in the target amplified video. Through the phase - based video motion amplification technology, the present invention can achieve high - precision micro - displacement measurement without contact; by combining the single - point high - resolution data provided by the long - focal - length camera and the large - range field of view provided by the short - focal - length camera, it can not only monitor large - range vibrations but also accurately measure local micro - displacements; using standard industrial camera equipment, non - contact measurement is realized, reducing the cost of the system and simplifying the installation process. Description of the Drawings

[0017] Figure 1 is a flowchart of a preferred embodiment of the method for monitoring the full - field vibration of a structure based on long - and short - focal - length cameras of the present invention; Figure 2 is a schematic diagram of a long - focal - length camera and a short - focal - length camera taking pictures of a target structure of the present invention; Figure 3 is a structural diagram of a preferred embodiment of the system for monitoring the full - field vibration of a structure based on long - and short - focal - length cameras of the present invention; Figure 4 is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Description of the Embodiment

[0018] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the following further elaborates on the present invention with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0019] With the development of Structural Health Monitoring (SHM), non-contact and high-precision measurement methods have become key requirements in many engineering and scientific applications. Traditional contact monitoring devices, such as accelerometers, vibration sensors, laser rangefinders, etc., although having high precision in many applications, still have several limitations. These traditional devices usually need to be in contact or close contact with the object to be measured, resulting in complex installation, additional maintenance costs, and possible interference to the object to be measured. In addition, in some scenarios that require non-contact, long-term monitoring or large-scale monitoring, traditional devices cannot provide flexible and efficient solutions.

[0020] Video monitoring technology, as an emerging non-contact measurement method, is increasingly applied in the field of structural health monitoring. Video monitoring technology captures the minute movements on the surface of an object through the collected video images, and uses image processing and analysis techniques to achieve precise measurement of motion trajectories, displacements, and vibrations. In particular, in vibration monitoring, video technology not only avoids the potential impact of physical contact on the object, but also can provide a wider range of application scenarios through computer vision processing. However, existing video monitoring technologies usually need to make a trade-off between large range and high resolution, and cannot have both large-range and high-resolution measurement capabilities at the same time. For example, the optical flow method is good at large-range monitoring but has insufficient resolution; the structured light method can provide high local resolution but cannot achieve large-range monitoring. In practical applications, for structural vibration monitoring and health assessment technologies, it is often required to have both large-range and high-resolution measurement capabilities at the same time.

[0021] Existing structural vibration monitoring technologies can be classified into the following categories: 1) Accelerometer and vibration sensor technology: Accelerometers are currently widely used devices in vibration monitoring and structural health monitoring. Accelerometers measure the acceleration changes of an object and then deduce the displacement or vibration. 2) Laser rangefinder and laser scanning technology: By irradiating the surface of an object with a laser beam and measuring the time difference of the reflected light to calculate the displacement of the object, laser technology usually has high measurement precision. 3) Computer vision and video motion analysis: Computer vision technology, especially video-based motion analysis, has gradually become a promising non-contact measurement technology. Existing video-based motion analysis methods are mainly carried out through the following technical means: 31) Optical flow method: By analyzing the pixel changes between video frames, estimate the motion and displacement of the object; 32) Structured light method and marker method: By projecting known light stripes or markers on the surface of the object, track the displacement of these light spots to calculate the displacement of the object.

[0022] The above-mentioned existing technologies have the following disadvantages: 1) Limitations and resolution issues: Although existing video-based motion analysis methods (such as optical flow methods) can monitor large-scale vibrations or displacements, their accuracy is insufficient, especially for measuring small displacements. For subtle motions or high-frequency vibrations, existing technologies are difficult to provide sufficient accuracy and reliability. 2) Equipment installation and environmental interference: Although devices such as laser rangefinders and accelerometers can provide high-precision data, their installation is complex and they are easily affected by environmental interference. For example, accelerometers must be in direct contact with the object being measured, which may affect the measurement results, especially in flexible structures or structures with large dynamic changes. 3) High requirements for lighting and texture: Many video-based monitoring methods rely on good lighting and texture features of the object surface, which pose significant limitations to certain environmental conditions (such as low light, reflection, or smooth surfaces), resulting in unstable measurement effects or decreased accuracy. 4) Inability to balance large range and high resolution simultaneously: Existing technologies usually make a trade-off between large range and high resolution. Some technologies are good at large-scale monitoring (such as optical flow methods), while others provide higher local resolution (such as structured light methods). However, in practical applications, the ability to measure both large range and high resolution is often required. 5) High cost and high complexity: Existing laser rangefinders, accelerometers, and complex computer vision algorithms usually require expensive equipment and complex post-processing, which increases the overall cost and usage threshold of the system.

[0023] To solve the above technical problems, the present invention provides a method for monitoring the full-field vibration of a structure based on long and short focal length cameras. It acquires local images obtained by the long focal length camera taking local pictures of the key parts of the target structure, and processes the local images to obtain the main vibration frequency of the target structure. It acquires the global video obtained by the short focal length camera taking global pictures of the target structure, and performs spatial decomposition processing on the video frame image signals corresponding to the global video by using complex-valued pyramid transform to obtain sub-band signals of different scales and different directions. It sets the parameters of the band-pass filter according to the main vibration frequency, and uses the band-pass filter to perform band-pass filtering on the phase time series corresponding to the sub-band signals to obtain the target phase time series, and performs amplification processing on the target phase time series to obtain a new phase time series. It performs phase modulation on the sub-band signals according to the new phase time series to obtain new sub-band signals, and performs video reconstruction according to the new sub-band signals of different scales and different directions to obtain the target amplified video. It performs marker point locking and vibration tracking on the target amplified video to obtain the vibration displacement signal of the marker point in the target amplified video. Through the phase-based video motion amplification technology, the present invention can achieve high-precision micro displacement measurement without contact; by combining the single-point high-resolution data provided by the long focal length camera and the large-range field of view provided by the short focal length camera, it can not only monitor large-range vibrations but also accurately measure local micro displacements; using standard industrial camera equipment, compared with traditional laser rangefinders and accelerometers, it realizes non-contact measurement, reduces the cost of the system, and simplifies the installation process.

[0024] The following further illustrates the application content by describing the embodiments in conjunction with the accompanying drawings.

[0025] A preferred embodiment of the method for monitoring the full-field vibration of a structure based on long and short focal length cameras according to the present invention is as Figure 1 shown, and specifically includes: S1. Acquire local images obtained by the long focal length camera taking local pictures of the key parts of the target structure, and process the local images to obtain the main vibration frequency of the target structure.

[0026] In an implementation manner of this embodiment, the acquiring local images obtained by the long focal length camera taking local pictures of the key parts of the target structure, and processing the local images to obtain the main vibration frequency of the target structure specifically includes: Acquire local images obtained by the long focal length camera taking local pictures of the key parts of the target structure where marker points are pasted; Use the corner detection method or the template matching method to lock the marker points in the local images; Track the spatial position change of the vibration of the marker points to obtain the initial vibration displacement signal; Perform a fast Fourier transform on the initial vibration displacement signal to obtain the spectral curve corresponding to the initial vibration displacement signal, and use the frequency corresponding to the point with the largest amplitude in the spectral curve as the main vibration frequency of the target structure.

[0027] Specifically, the present invention combines the characteristics of a long-focus camera and a short-focus camera, makes full use of the high-resolution images obtained by the long-focus camera in a local area, extracts the fundamental frequency information of the structure as the basis for filter parameters, and then performs phase-based video processing on the wide-field video stream captured by the short-focus camera. The structural fundamental frequency data is obtained by the long-focus camera taking local pictures of the key parts of the target structure (such as beam ends, supports, nodes, etc.). If the shooting position has good texture features, it can be directly shot; otherwise, artificial fiducial points can be pasted. In this way, the image is stable and has a high contrast, which is beneficial for subsequent tracking and analysis. The process of obtaining the structural fundamental frequency data includes: using corner detection (such as Harris corner detection) or template matching to lock the fiducial points, tracking the spatial position of the vibration of the fiducial points to obtain a time series signal (i.e., the initial vibration displacement signal), using the fast Fourier transform (FFT, Fast Fourier Transform) to analyze and obtain the spectral characteristics (i.e., the spectral curve) of the time series signal of the vibration of the fiducial points during the measurement period, and clearly extracting the main vibration frequency of the structure (i.e., the structural fundamental frequency data) from the spectral curve, which is used as an important basis for selecting the video motion amplification frequency band, that is, selecting the filter bandwidth for processing the short-focus camera video stream based on the structural fundamental frequency data extracted by the long-focus camera. As Figure 2 shown, the present invention uses the long-focus camera 1 to take local pictures of the key parts of the target structure 3 with fiducial points pasted thereon to obtain local images (i.e., high-resolution images), and uses the short-focus camera 2 to take global pictures of the target structure 3 to obtain global videos (i.e., large-range videos / wide-field videos). Among them, the long-focus camera 1 and the short-focus camera 2 work synchronously, and data synchronization acquisition is ensured through timing control.

[0028] S2. Obtain the global video obtained by the short-focus camera taking a global picture of the target structure, and perform spatial decomposition processing on the video frame image signal corresponding to the global video using the complex-valued pyramid transform to obtain sub-band signals of different scales and different directions.

[0029] In an implementation manner of this embodiment, the obtaining the global video obtained by the short-focus camera taking a global picture of the target structure, and performing spatial decomposition processing on the video frame image signal corresponding to the global video using the complex-valued pyramid transform to obtain sub-band signals of different scales and different directions specifically includes: Obtain the global video obtained by the short-focus camera taking a global picture of the target structure, and extract video frames from the global video to obtain a video frame image sequence corresponding to the global video; Construct a time-domain signal that changes over time according to the intensity change of each pixel in the video frame image sequence and in the order of the video frame images in the video frame image sequence; Perform a Fourier transform on the time-domain signal to obtain the video frame image signal corresponding to the global video, that is, first perform a frequency-domain analysis on the image signal (referring to the time-domain signal) through the Fourier transform. The intensity of a two-dimensional image can be expressed as the superposition of frequency components by the Fourier series: ; where, represents the video frame image signal, represents the horizontal axis component of the pixel coordinate , represents the pixel coordinate 's vertical axis component, represents time, represents the horizontal axis component of the spatial frequency , represents the vertical axis component of the spatial frequency , represents the frequency component, represents the amplitude response of the video frame image signal, represents the phase change of the video frame image signal, represents the imaginary unit; Perform a multi-scale spatial decomposition process on the video frame image signal using a complex-valued pyramid transform to obtain sub-band signals of different scales and different directions. This pyramid structure is a multi-scale, multi-directional band-pass filter bank that can represent the amplitude information and phase information of an image (referring to the sub-band signal. It should be noted that the sub-band signal in the present invention can be understood as a sub-band image, and the video frame image signal can also be understood as a video frame image) in complex form. The expression of the sub-band signal for each frequency component is: ; where, represents the sub-band signal corresponding to the frequency component ; represents the amplitude information of the sub-band signal corresponding to the frequency component , indicating the intensity of the image pixel at the frequency component ; represents the phase time series corresponding to the sub-band signal; represents the pixel displacement caused by image motion (i.e., the motion of the marked points in the image) that changes over time.

[0030] Specifically, the core idea of the phase motion amplification method is to regard the intensity change of each pixel in the image sequence as a time-domain signal varying with time, use Fourier analysis technology to convert the motion signal of the pixel (referring to the above time-domain signal, that is, the pixel intensity change signal) to the frequency domain, and realize the amplification of the motion signal by analyzing the phase information of the frequency-domain signal, so as to effectively enhance the tiny motion signal. It should be noted that when an object undergoes tiny motions (such as vibration, deformation), its projected position in the image will change with time, resulting in the change of the brightness (intensity) at the same pixel position due to the movement of the object or the change of light. Since it is difficult to directly measure sub-pixel motion, the change of intensity with time is regarded as an indirect representation of motion, and the intensity change signal implies displacement information.

[0031] S3. Set the parameters of the band-pass filter according to the main vibration frequency, and use the band-pass filter to perform band-pass filtering on the phase time series corresponding to the sub-band signal to obtain a target phase time series, and perform amplification processing on the target phase time series to obtain a new phase time series.

[0032] In an implementation manner of this embodiment, the step of setting the parameters of the band-pass filter according to the main vibration frequency, using the band-pass filter to perform band-pass filtering on the phase time series corresponding to the sub-band signal to obtain a target phase time series, and performing amplification processing on the target phase time series to obtain a new phase time series specifically includes: Set the parameters of the band-pass filter according to the main vibration frequency, set the main vibration frequency as the center frequency of the band-pass filter, and set the frequency band with a set length centered on the main vibration frequency as the bandwidth of the band-pass filter; Use the band-pass filter with the set parameters to perform band-pass filtering on the phase time series corresponding to the sub-band signal to obtain a target phase time series: ; where represents the target phase time series corresponding to the frequency component ; Perform amplification processing on the target phase time series, that is, multiply the filtered phase by an amplification factor (i.e., amplification coefficient) to obtain a new phase time series: ; where represents the new phase time series corresponding to the frequency component , represents the amplification factor, is greater than 0 and is used to enhance the motion.

[0033] Specifically, the core of the present invention lies in extracting minute motion information from the local phase change of an image. According to the Fourier shift theorem, if an image undergoes translation, its phase in the complex domain representation will change linearly accordingly. Therefore, the phase of the sub-band signal contains displacement information. To highlight the motion components at a specific frequency while suppressing low-frequency drift and high-frequency noise, band-pass filtering is performed on the phase time series of each point (i.e., pixel). When applying phase filtering to the global video of a short-focus camera, the parameters of the time-domain band-pass filter are set as follows: the center frequency of the short-focus camera (global video processing) , ; the bandwidth of the short-focus camera (global video processing) is selected. Generally speaking, a relatively narrow frequency band near the main peak frequency (i.e., the main vibration frequency ) is used as the amplified frequency band to ensure that the focus of motion amplification is concentrated on the real dynamic response area of the structure and to avoid the amplification of noise or other irrelevant frequency components. The typical value is .

[0034] S4. Phase-modulate the sub-band signal according to the new phase time series to obtain a new sub-band signal, and perform video reconstruction based on the new sub-band signals of different scales and different directions to obtain the target amplified video.

[0035] In an implementation manner of this embodiment, the phase-modulating the sub-band signal according to the new phase time series to obtain a new sub-band signal, and performing video reconstruction based on the new sub-band signals of different scales and different directions to obtain the target amplified video specifically includes: Phase-modulate the sub-band signal according to the new phase time series to obtain a new sub-band signal: ; wherein, represents the new sub-band signal corresponding to the frequency component ; Perform video reconstruction on the new sub-band signals of different scales and different directions to obtain the target amplified video: ; wherein, represents the amplified video frame image signal corresponding to the target amplified video.

[0036] Specifically, after performing phase adjustment on sub-bands of all scales and directions, the image frames at each moment are re-synthesized (i.e., magnifying the video frame images), and the entire video sequence (i.e., the target magnified video) is reconstructed in reverse. This step synthesizes the sub-band signals of each frequency component to obtain the final motion magnified video. Through the phase changes in the video sequence, extremely tiny displacement information can be extracted. Combining with the video motion magnification technology, these tiny displacements become obvious, thus providing high-precision measurement data without contacting the object. Compared with the accuracy of traditional accelerometers or laser rangefinders, the video motion magnification technology can provide more detailed and efficient displacement monitoring.

[0037] S5. Perform landmark locking and vibration tracking on the target magnified video to obtain the vibration displacement signal of the landmark in the target magnified video.

[0038] In an implementation manner of this embodiment, the performing landmark locking and vibration tracking on the target magnified video to obtain the vibration displacement signal of the landmark in the target magnified video specifically includes: Use the corner detection method or the template matching method to lock the landmark in the magnified video frame image corresponding to the target magnified video; Track the spatial position change of the landmark vibration to obtain the vibration displacement signal; Obtain the structural full-field vibration monitoring result according to the vibration displacement signal; Wherein, the magnified video frame image corresponds to the magnified video frame image signal.

[0039] In an implementation manner of this embodiment, before using the corner detection method or the template matching method to lock the landmark in the magnified video frame image corresponding to the target magnified video, it further includes: Intercept the region of interest containing the key part from the magnified video frame image corresponding to the target magnified video according to the key part.

[0040] Specifically, first, perform ROI (Region of Interest) selection on the processed motion magnification video (i.e., the target magnification video). According to the key parts (i.e., the key vibration response regions selected according to the design drawing or experience), intercept the ROI region from the magnified video frame images corresponding to the target magnification video. Then, use corner detection (such as Harris corner detection) or template matching to lock the landmark points in the ROI region, track the spatial positions of the vibrations of the landmark points, obtain a new time series signal (i.e., the vibration displacement signal), and further obtain the vibration information at the positions where the landmark points are pasted on the target structure (i.e., the key parts), so that the tiny vibration information of the structure that is difficult to detect in the large field of view of the original short-focus camera can be accurately extracted. It should be noted that in step S4, the sub-bands of different frequency components can also be directly synthesized into the magnified video frame images, and in step S5, image processing is directly performed on the magnified video frame images to obtain the vibration displacement signal, that is, skipping the video synthesis step.

[0041] The present invention addresses the problems of poor non-contact performance, limited measurement range, insufficient resolution, etc. existing in the current structural vibration monitoring and health assessment technologies, and proposes a method for full-field vibration monitoring of structures based on long- and short-focus cameras. This method combines the characteristics of long- and short-focus cameras, makes full use of the high-resolution images obtained by the long-focus camera in a local area, extracts the fundamental frequency information of the structure as the basis for filter parameters, and then performs phase-based video processing on the wide-field video stream captured by the short-focus camera. The filter bandwidth for processing the data stream of the short-focus camera is selected based on the fundamental frequency information of the structure extracted by the long-focus camera, and the local phase change information in the image is analyzed to accurately extract the tiny displacement signal of the structure in a specific frequency band, effectively separating the dynamic response of the target structure from the background interference, and realizing the magnification of tiny vibrations and the extraction of displacements. The present invention uses the long-focus camera to provide single-point high-resolution data and the short-focus camera to provide a large field of view. Through the fusion of the long-focus camera and the short-focus camera, large-range monitoring and high-precision local measurement are simultaneously achieved, avoiding the trade-off between these two aspects in the prior art. Image data is remotely collected by camera devices without contacting the object to be measured, avoiding the interference brought by traditional sensors and realizing true non-contact measurement. Non-contact measurement ensures that the true response of the structure is not affected by external interference and can monitor any irregularly shaped or sensitive structure for a long time under dynamic conditions. For large-scale structures (such as bridges, buildings, etc.) or complex environments (such as bad weather, dangerous areas), this technology provides greater flexibility and operability.

[0042] Compared with traditional contact or local monitoring technologies such as accelerometers and laser rangefinders, the present invention does not require the installation of traditional contact sensors (such as accelerometers and laser rangefinders) on the surface of the structure, and can achieve long-distance, non-contact, and high-precision micro-vibration monitoring of large structures under actual working conditions. Through the fusion of long- and short-focus cameras and the phase-based video motion amplification technology, the present invention can comprehensively monitor a large-scale structure under various environmental conditions, while providing high-resolution detailed data, significantly improving the detection accuracy of micro-displacements.

[0043] The technical advantages of the present invention are mainly reflected in the non-contact detection method, simple deployment, strong adaptability, suitable for a variety of complex environments and large-scale structure scenarios, and can achieve true non-contact measurement; by fusing video information of different focal lengths, it breaks through the traditional trade-off limit between large-scale monitoring and high-resolution measurement, and while ensuring the monitoring coverage, improves the accuracy and timeliness of local measurement; through the phase-based video motion amplification technology, it can effectively amplify micro-displacements and improve the detection accuracy.

[0044] In addition, based on the above-mentioned method for full-field vibration monitoring of a structure using long- and short-focus cameras, the present invention also provides a system for full-field vibration monitoring of a structure using long- and short-focus cameras, wherein a preferred embodiment of the system for full-field vibration monitoring of a structure using long- and short-focus cameras is as Figure 3 shown, and specifically includes: Main vibration frequency acquisition module 01: used to acquire a local image obtained by the long-focus camera for local shooting of the key part of the target structure, and process the local image to obtain the main vibration frequency of the target structure; Sub-band signal acquisition module 02: used to acquire a global video obtained by the short-focus camera for global shooting of the target structure, and perform spatial decomposition processing on the video frame image signal corresponding to the global video by using complex-valued pyramid transform to obtain sub-band signals of different scales and different directions; New phase time series acquisition module 03: used to set the parameters of the band-pass filter according to the main vibration frequency, and perform band-pass filtering processing on the phase time series corresponding to the sub-band signals by using the band-pass filter to obtain a target phase time series, and perform amplification processing on the target phase time series to obtain a new phase time series; Target amplified video acquisition module 04: used to perform phase modulation on the sub-band signals according to the new phase time series to obtain new sub-band signals, and perform video reconstruction according to the new sub-band signals of different scales and different directions to obtain a target amplified video; Vibration displacement signal acquisition module 05: used to perform landmark locking and vibration tracking on the target amplified video to obtain the vibration displacement signal of the landmark in the target amplified video.

[0045] In addition, based on the above-mentioned method and system for structural full-field vibration monitoring based on long and short focal length cameras, the present invention also correspondingly provides a terminal. In a preferred embodiment of the terminal, as Figure 4 shown, it specifically includes a processor 10, a memory 20, and a display 30. Figure 4 Only some components of the terminal are shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0046] The memory 20 can be an internal storage unit of the terminal in some embodiments, such as the hard disk or memory of the terminal. The memory 20 can also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, and a Flash Card equipped on the terminal, etc. Further, the memory 20 can also include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software installed on the terminal and various types of data, such as storing the program code of the terminal, etc. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, a program 40 for structural full-field vibration monitoring based on long and short focal length cameras is stored on the memory 20, and the program 40 for structural full-field vibration monitoring based on long and short focal length cameras can be executed by the processor 10, thereby implementing the steps of the method for structural full-field vibration monitoring based on long and short focal length cameras in the present application.

[0047] The processor 10 can be a Central Processing Unit (CPU), a microprocessor, or other data processing chips in some embodiments, and is used to run the program code stored in the memory 20 or process data, such as executing the program 40 for structural full-field vibration monitoring based on long and short focal length cameras, etc.

[0048] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) toucher, etc. in some embodiments. The display 30 is used to display information on the terminal and to display a visual user interface.

[0049] In one embodiment, when the processor 10 executes the program 40 for structural full-field vibration monitoring based on long and short focal length cameras stored in the memory 20, the steps of the method for structural full-field vibration monitoring based on long and short focal length cameras as described above are implemented.

[0050] The present invention also correspondingly provides a computer-readable storage medium. The computer-readable storage medium stores a program for structural full-field vibration monitoring based on a long-short focal length camera. When the program for structural full-field vibration monitoring based on a long-short focal length camera is executed by a processor, the steps of the method for structural full-field vibration monitoring based on a long-short focal length camera as described above are implemented.

[0051] It should be noted that in this text, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or terminal. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or terminal including that element.

[0052] Certainly, those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When the program is executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disc, etc.

[0053] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description. All such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A method for monitoring full-field vibration of a structure based on a long- and short-focus camera, characterized in that: The method for monitoring the full-field vibration of a structure based on a long- and short-focus camera comprises: Acquire a local image obtained by partially photographing a key part of a target structure with a telephoto camera, and process the local image to obtain a main vibration frequency of the target structure; A global video is obtained by taking a global shot of the target structure by a short-focus camera, and a video frame image signal corresponding to the global video is spatially decomposed by using a complex-valued pyramid transform to obtain sub-band signals of different scales and directions; The parameters of the bandpass filter are set according to the main vibration frequency, and the phase time series corresponding to the sub-band signal is subjected to bandpass filtering processing by using the bandpass filter to obtain a target phase time series, and the target phase time series is amplified to obtain a new phase time series; Phase modulating the sub-band signal according to the new phase time sequence to obtain a new sub-band signal, and reconstructing the video according to the new sub-band signals of different scales and directions to obtain a target magnified video; The target magnified video is subjected to marker point locking and vibration tracking to obtain a vibration displacement signal of the marker point in the target magnified video.

2. The method for monitoring full-field vibration of a structure based on a long-short focal length camera according to claim 1, characterized in that: The acquiring of a local image obtained by locally photographing a key part of the target structure with a telephoto camera, and processing the local image to obtain the main vibration frequency of the target structure specifically includes: Acquire a local image obtained by partially photographing a key part of the target structure to which the marker point is attached using a telephoto camera; Using a corner point detection method or a template matching method to lock the marker point in the local image; Tracking the spatial position change of the vibration of the mark point to obtain an initial vibration displacement signal; The initial vibration displacement signal is subjected to a fast Fourier transform to obtain a frequency spectrum curve corresponding to the initial vibration displacement signal, and the frequency corresponding to the maximum amplitude point in the frequency spectrum curve is taken as the main vibration frequency of the target structure.

3. The method for monitoring the full-field vibration of a structure based on a long-short focal length camera according to claim 1, characterized in that: The acquiring of a global video obtained by globally photographing the target structure with a short-focus camera, and the spatial decomposition processing of the video frame image signal corresponding to the global video by using a complex-valued pyramid transform to obtain sub-band signals of different scales and directions specifically include: Acquire a global video obtained by globally photographing the target structure with a short-focus camera, and extract video frames from the global video to obtain a video frame image sequence corresponding to the global video; According to the intensity change of each pixel in the video frame image sequence and in accordance with the order of the video frame images in the video frame image sequence, constructing a time domain signal that changes with time; Perform Fourier transform on the time domain signal to obtain a video frame image signal corresponding to the global video: ； in, Represents the video frame image signal, Represents the horizontal axis component of the pixel coordinate, Represents the vertical axis component of the pixel coordinate, Indicates time, The horizontal axis component represents the spatial frequency, The vertical axis represents the spatial frequency component, represents the frequency component, represents the amplitude response of the video frame image signal, Indicates the phase change of the video frame image signal, represents an imaginary unit; The video frame image signal is subjected to multi-scale spatial decomposition processing using a complex-valued pyramid transform to obtain sub-band signals of different scales and directions: ； in, Indicates frequency components The corresponding subband signal is Represents the amplitude information of the subband signal, represents the phase time series corresponding to the subband signal, Indicates pixel displacement.

4. The method for monitoring full-field vibration of a structure based on a long-short focal length camera according to claim 3, characterized in that: The step of setting the parameters of the bandpass filter according to the main vibration frequency, and using the bandpass filter to perform bandpass filtering on the phase time series corresponding to the subband signal to obtain a target phase time series, and amplifying the target phase time series to obtain a new phase time series specifically includes: Setting parameters of a bandpass filter according to the main vibration frequency, setting the main vibration frequency as the center frequency of the bandpass filter, and setting a set length frequency band centered on the main vibration frequency as the bandwidth of the bandpass filter; The phase time series corresponding to the sub-band signal is subjected to band-pass filtering by using the band-pass filter with set parameters to obtain the target phase time series: ； in, Indicates frequency components The corresponding target phase time series; The target phase time series is amplified to obtain a new phase time series: ； in, Indicates frequency components The corresponding new phase time series, Represents the magnification factor.

5. The method for monitoring full-field vibration of a structure based on a long-short focal length camera according to claim 4, characterized in that: The phase modulation of the sub-band signal according to the new phase time sequence to obtain a new sub-band signal, and the video reconstruction according to the new sub-band signals of different scales and different directions to obtain the target magnified video specifically includes: The sub-band signal is phase modulated according to the new phase time sequence to obtain a new sub-band signal: ； in, Indicates frequency components The corresponding new sub-band signal; The new sub-band signals of different scales and directions are reconstructed through video to obtain the target enlarged video: ； in, Indicates the enlarged video frame image signal corresponding to the target enlarged video.

6. The method for monitoring full-field vibration of a structure based on a long-short focal length camera according to claim 5, characterized in that: The step of performing marker point locking and vibration tracking on the target magnified video to obtain a vibration displacement signal of the marker point in the target magnified video specifically includes: Using a corner point detection method or a template matching method to lock the landmark points in the enlarged video frame image corresponding to the target enlarged video; Tracking the spatial position change of the vibration of the mark point to obtain a vibration displacement signal; According to the vibration displacement signal, a full-field vibration monitoring result of the structure is obtained; The enlarged video frame image corresponds to the enlarged video frame image signal.

7. The method for monitoring full-field vibration of a structure based on a long-short focal length camera according to claim 6, characterized in that: The method of using a corner point detection method or a template matching method to lock the landmark points in the enlarged video frame image corresponding to the target enlarged video also includes: According to the key part, a region of interest including the key part is captured from the enlarged video frame image corresponding to the target enlarged video.

8. A structural full-field vibration monitoring system based on long- and short-focus cameras, characterized in that: The structural full-field vibration monitoring system based on long- and short-focus cameras includes: Main vibration frequency acquisition module: used to acquire a local image obtained by partially photographing a key part of a target structure with a telephoto camera, and process the local image to obtain the main vibration frequency of the target structure; Sub-band signal acquisition module: used to acquire a global video obtained by globally photographing the target structure with a short-focus camera, and perform spatial decomposition processing on the video frame image signal corresponding to the global video using a complex-valued pyramid transform to obtain sub-band signals of different scales and directions; New phase time series acquisition module: used to set the parameters of the bandpass filter according to the main vibration frequency, and use the bandpass filter to perform bandpass filtering on the phase time series corresponding to the sub-band signal to obtain a target phase time series, and amplify the target phase time series to obtain a new phase time series; A target magnified video acquisition module is used to perform phase modulation on the sub-band signal according to the new phase time sequence to obtain a new sub-band signal, and to perform video reconstruction according to the new sub-band signals of different scales and directions to obtain a target magnified video; The vibration displacement signal acquisition module is used to lock the mark point and track the vibration of the target magnified video to obtain the vibration displacement signal of the mark point in the target magnified video.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a structural full-field vibration monitoring program based on a long- and short-focus camera stored in the memory and executable on the processor. When the structural full-field vibration monitoring program based on a long- and short-focus camera is executed by the processor, the steps of the structural full-field vibration monitoring method based on a long- and short-focus camera as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a structural full-field vibration monitoring program based on long- and short-focus cameras. When the structural full-field vibration monitoring program based on long- and short-focus cameras is executed by a processor, the steps of the structural full-field vibration monitoring method based on long- and short-focus cameras as described in any one of claims 1 to 7 are implemented.

Images