Visual vibration measurement method, system and device fusing super-resolution and interpolation technology

Abstract

The present application belongs to the technical field of image processing, and particularly relates to a visual vibration measurement method, system and device fusing super-resolution and frame interpolation technology. The method collects the video of the measured object by a non-contact industrial camera, uses frame interpolation technology to improve the video frame rate to meet the Nyquist sampling theorem, and solves the problem of high-frequency vibration information loss. The super-resolution technology is used to enhance the image resolution, overcome uneven illumination and background noise interference, and improve the detection accuracy of micro displacement. The system includes video acquisition, frame interpolation processing, super-resolution processing, optical flow analysis and spectrum analysis modules, and realizes accurate extraction of vibration characteristic frequency and amplitude. The method does not require additional sensors, reduces hardware cost, and is suitable for equipment monitoring under various complex working conditions.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a visual vibration measurement method, system and device fusing super-resolution and frame interpolation technology. BACKGROUND

[0002] At present, vibration measurement of industrial equipment mainly includes two methods of contact type and non-contact type. The contact type measurement usually adopts acceleration sensors, strain gauges and the like directly installed on the surface of the measured equipment to collect vibration data through electrical signals. This kind of method is mature in technology, but needs physical contact, which may introduce additional mass load and affect the vibration characteristics of the measured object. The non-contact measurement technology includes laser Doppler vibrometer, high-speed camera visual measurement and the like, which avoids the contact problem, but has high requirements for equipment installation environment and is expensive.

[0003] The existing vibration measurement technology has the following main problems:

[0004] (1) The contact type measurement changes the vibration characteristics of the measured object, and the measurement error is significant especially for light structure or precision equipment;

[0005] (2) The non-contact method (such as laser vibration measurement) is expensive, and has strict requirements for the test environment, which is difficult to be widely applied in complex industrial scenes;

[0006] (3) The traditional visual vibration measurement method is limited by the frame rate and resolution of the camera, and it is difficult to meet the accurate measurement demand of high-frequency vibration, and is easily disturbed by factors such as light change and background noise. These problems seriously restrict the application effect and promotion value of the vibration measurement technology in the industrial monitoring field. SUMMARY

[0007] The purpose of the present application is to provide a visual vibration measurement method, system and device fusing super-resolution and frame interpolation technology, which can overcome the problems of low vibration measurement precision and poor anti-interference ability caused by insufficient video frame rate and resolution in the existing visual vibration measurement technology without contacting the measured equipment.

[0008] The present application realizes the above-mentioned purpose through the following technical solutions:

[0009] In a first aspect, the present application provides a visual vibration measurement method fusing super-resolution and frame interpolation technology, which comprises the following steps:

[0010] S1, collecting an original video sequence of a measured object , and the original frame rate of the sequence is F; wherein is the i-th frame image, and the time domain covers , , is the frame sampling interval;

[0011] S2, setting a frame rate object in the original video sequence to perform the interpolation processing, forming an interpolated video sequence satisfying the Nyquist sampling theorem requirement after interpolation ; wherein is the frame image after interpolation, N is the number of frames of the original video sequence, is the number of frames of the video sequence after interpolation, is the index of the original video frame, m is the index of the video frame after interpolation, is the interpolation ratio;

[0012] S3, obtaining the video sequence to be analyzed after super-resolution processing ; wherein is a high-resolution frame;

[0013] S4, extracting the pixel motion information of the super-resolution video sequence of the target vibration region of the measured object to convert into a vibration time domain signal ;

[0014] S5, according to the vibration time domain signal , determining the vibration characteristic frequency and amplitude of the measured object.

[0015] Further, step S2 includes:

[0016] S21, calculating the optical flow field of adjacent frames in the set frame rate object , and the optical flow calculation satisfies the constraint equation: ; wherein is the gray value of the frame image at the pixel point, is the optical flow displacement of the pixel;

[0017] S22, after estimating the pixel motion trajectory through the optical flow field , constructing an interpolation model to generate an intermediate frame : , wherein is a fusion weight function, ;

[0018] S23, after interpolation based on the intermediate frame , the video sequence to be analyzed is obtained as , and the new frame rate is .

[0019] Further, step S3 includes:​

[0020] S31. Video sequence to be analyzed Each resolution frame to be processed in Constructing the energy function Used to describe the reconstruction relationship between a low-resolution image and a target high-resolution image: ;in For downsampling operators, To reconstruct the error term, For regularization terms, This is the balance coefficient;

[0021] S32. Minimize the energy function using an iterative optimization algorithm. Recover the preset resolution frame ;

[0022] S33. Repeat steps S31-S32 to generate a super-resolution video sequence. ;

[0023] Among them, the resolution frames to be processed , Image size, preset resolution frame , This is the super-resolution magnification.

[0024] Furthermore, the iterative optimization algorithm employs gradient descent, conjugate gradient, or convex optimization algorithms.

[0025] Furthermore, step S4 includes:

[0026] S41. Calculate super-resolution video sequences Medium continuous frames Optical flow field Get each pixel instantaneous velocity ;

[0027] S42. Determine the vibration region of the target object and calculate the average velocity of all pixels within that region, including: , It is the number of pixels in the target area. The average speed of movement in the region;

[0028] S43. Based on the average velocity over the time series and Determine the vibration time-domain signal .

[0029] Furthermore, step S5 includes:

[0030] S51. Convert the time-domain signal into a frequency-domain signal using fast Fourier transform, and extract the frequency component with the largest amplitude in the frequency-domain signal as the vibration characteristic frequency of the object under test.

[0031] S52. Determine the vibration amplitude A based on the amplitude of the frequency domain signal at the vibration characteristic frequency, combined with the pre-calibrated coefficient k.

[0032] Furthermore, the original video sequence of the object under test is acquired through a video acquisition device that has no physical connection with the device under test, so as to achieve non-contact image acquisition.

[0033] Secondly, the present invention proposes a visual vibration measurement system that integrates super-resolution and frame interpolation techniques to implement the visual vibration measurement method described above. The system includes:

[0034] The video acquisition module is used to acquire the original video sequence of the object under test through a video acquisition device that has no physical connection with the device under test. The original frame rate of the sequence is F; where For the first Frame image, For frame sampling interval, time domain coverage ;

[0035] The frame interpolation module is used to process the original video sequence. The frame rate object is set in the middle to perform frame interpolation processing, forming a video sequence to be analyzed that satisfies the Nyquist sampling theorem after interpolation. The frame rate after sequence interpolation is ;in For the first frame after interpolation Frame image, where N is the number of frames in the original video sequence. This represents the number of frames in the video sequence after frame interpolation. is the index of the original video frame, and m is the index of the video frame after interpolation. The frame interpolation ratio;

[0036] The super-resolution processing module is used to acquire the video sequence to be analyzed. Super-resolution video sequence after super-resolution processing ;in High-resolution frames;

[0037] The optical flow analysis module is used to extract super-resolution video sequences of the vibration region of the target object. The pixel motion information is converted into vibration time-domain signals. ;

[0038] The spectrum analysis module is used to analyze the vibration time-domain signal. Determine the vibration characteristic frequency of the object being measured. and amplitude .

[0039] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the visual vibration measurement method as described above when executing the computer program.

[0040] In a fourth aspect, the present application provides a computer readable storage medium storing a computer program, wherein the computer program implements the steps of the visual vibration measurement method as described above when executed by a processor.

[0041] The present application has the following beneficial effects:

[0042] The present application effectively overcomes the limitations of traditional measurement methods through non-contact vibration measurement. In terms of technology, the present application uses frame interpolation technology to improve the video frame rate, ensuring that the sampling process meets the Nyquist theorem and solving the problem of high-frequency vibration information loss caused by low-frequency sampling. At the same time, super-resolution technology is used to enhance image resolution, significantly improving the detection accuracy of small displacements and providing a more reliable image basis for subsequent vibration analysis. In terms of application, this method eliminates the dependence on physical sensors and avoids the common quality load effect in contact measurement, making it suitable for equipment monitoring under various complex working conditions. Compared with existing technologies, the present application significantly reduces hardware costs and environmental requirements, and ordinary industrial cameras can meet the measurement requirements while effectively suppressing background noise interference. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 A flowchart of a visual vibration measurement method combining super-resolution and frame interpolation technology provided by an embodiment of the present application;

[0044] Figure 2 Another flowchart of a visual vibration measurement method combining super-resolution and frame interpolation technology provided by an embodiment of the present application;

[0045] Figure 3 A system block diagram of a visual vibration measurement system combining super-resolution and frame interpolation technology provided by an embodiment of the present application;

[0046] Figure 4 A time-domain displacement signal curve graph when the video acquisition original frame rate is 30fps in the experimental case of the specific embodiment of the present application;

[0047] Figure 5 A frequency-domain spectral distribution graph when the video acquisition original frame rate is 30fps in the experimental case of the specific embodiment of the present application;

[0048] Figure 6 A time-domain displacement signal curve graph when the video is interpolated to 80fps in the experimental case of the specific embodiment of the present application;

[0049] Figure 7 Time domain displacement signal curve of video interpolation to 120fps in the experimental case of the specific embodiment of the present application;

[0050] Figure 8 Time domain displacement signal curve of video interpolation to 120fps in the experimental case of the specific embodiment of the present application;

[0051] Figure 9 Time domain displacement signal curve of video interpolation to 120fps in the experimental case of the specific embodiment of the present application;

[0052] Figure 10 Time domain displacement signal curve of video interpolation to 120fps in the experimental case of the specific embodiment of the present application;

[0053] Figure 11 Time domain displacement signal curve of video interpolation to 120fps in the experimental case of the specific embodiment of the present application;

[0054] Figure 12 Time domain displacement signal curve of video interpolation to 120fps in the experimental case of the specific embodiment of the present application;

[0055] Figure 13 Time domain displacement signal curve of video interpolation to 120fps in the experimental case of the specific embodiment of the present application. DETAILED DESCRIPTION

[0056] It is necessary to point out here that the following detailed description is only used to further illustrate the present application and cannot be understood as limiting the protection scope of the present application. Those skilled in the art can make some non-essential improvements and adjustments to the present application according to the above application content.

[0057] Embodiment 1

[0058] Please combine Figure 1 and Figure 2 , a visual vibration measurement method combining super-resolution and interpolation technology is proposed in a specific embodiment of the present application, and the method comprises the following steps:

[0059] S1, collecting the original video sequence of the measured object , the original frame rate of the sequence is F; wherein is the frame image, covering , The frame sampling interval is used to obtain the original video sequence of the object under test through a video acquisition device that has no physical connection with the object under test (such as industrial equipment that needs to be diagnosed) to achieve non-contact image acquisition.

[0060] S2, For the original video sequence The frame rate object is set in the middle to perform frame interpolation processing, forming a video sequence to be analyzed that satisfies the Nyquist sampling theorem after interpolation. The frame rate after sequence interpolation is ;in For the first frame after interpolation Frame image, where N is the number of frames in the original video sequence. This represents the number of frames in the video sequence after frame interpolation. is the index of the original video frame, and m is the index of the video frame after interpolation. This represents the frame interpolation ratio.

[0061] Traditional visual vibration measurement methods often suffer from insufficient camera frame rates, leading to the loss of high-frequency vibration information and failing to meet the Nyquist sampling theorem (the sampling frequency must be greater than twice the highest frequency of the signal). Therefore, this application employs frame interpolation technology. By calculating the optical flow field between adjacent frames, the pixel motion trajectory is estimated, and an interpolation model is constructed to generate intermediate frames, thereby increasing the video frame rate. This process ensures that the interpolated video sequence meets the Nyquist sampling theorem requirements, accurately captures high-frequency vibration information, and avoids information loss.

[0062] Due to the interpolated video Due to physical limitations of imaging equipment and transmission losses, there is a problem of insufficient resolution. Super-resolution can recover high-resolution details by learning complementary information between low-resolution frames.

[0063] S3. Obtain the video sequence to be analyzed. Super-resolution video sequence after super-resolution processing ;in It is a high-resolution frame.

[0064] S4. Extract the super-resolution video sequence of the vibration region of the target object. The pixel motion information is converted into vibration time-domain signals. .

[0065] S5. Based on the vibration time domain signal Determine the vibration characteristic frequency of the object being measured. and amplitude .

[0066] In the present application, the visual vibration measurement method does not need to install vibration sensors on the surface of the measured object, and only needs to use an ordinary industrial camera to realize vibration measurement, which can be applied to scenes where it is difficult to install sensors or is not allowed to interfere with the measured object. In addition, the original video of the measured object in the present application can be video information of a vibration table, or video information of various production equipment such as motors, pumps, and reduction boxes.

[0067] It can be understood that the present application combines the interpolation frame technology with the super-resolution processing, the interpolation frame processing ensures that the video sampling rate meets the Nyquist requirement, effectively avoiding the loss of high-frequency vibration information; the super-resolution technology significantly improves the image detail resolution, and provides a more accurate pixel motion basis for optical flow analysis. Secondly, the method establishes a complete "video acquisition-frame rate improvement-resolution enhancement-vibration extraction" technical chain, which completely eliminates the mass load effect caused by sensor installation through non-contact measurement method, and reduces the performance requirements of hardware equipment.

[0068] Further preferably, the step S2 comprises:

[0069] S21, calculating the optical flow field of adjacent frames in the set frame rate object . The optical flow calculation satisfies the constraint equation: ; wherein is the gray value of the pixel point in the first frame image, is the optical flow displacement of the pixel.

[0070] S22, after estimating the pixel motion trajectory through the optical flow field , constructing an interpolation frame model to generate an intermediate frame : , wherein is a fusion weight function, .

[0071] S23, after interpolation based on the intermediate frame , the video sequence to be analyzed is obtained as , and the new frame rate is .

[0072] It should be noted that the "set frame rate object" in the present application refers to a specific video segment or the entire video sequence selected for interpolation processing to improve the frame rate in the original video sequence.

[0073] Specifically, in the visual vibration measurement method, because the frame rate of the original video sequence does not meet the requirement of the Nyquist sampling theorem, leading to the loss of high-frequency vibration information, it is necessary to perform interpolation processing on the video sequence.

[0074] ​In step S21, the optical flow field of adjacent frames in the frame rate setting object is calculated by an algorithm. In this process, the gray value and optical flow displacement of each pixel point are accurately captured to meet the constraint equation of optical flow calculation and ensure the accuracy of motion estimation. Subsequently, in step S22, the system estimates the motion trajectory of the pixel using the optical flow field and intelligently generates the intermediate frame through the constructed frame interpolation model. In this process, the application of the fusion weight function makes the generation of the intermediate frame smoother and more consistent with the actual motion law, effectively improving the frame interpolation quality. Finally, in step S23, based on the generated intermediate frame, the frame interpolation processing is performed to obtain a new video sequence that meets the Nyquist sampling theorem requirement. The improvement of the new frame rate ensures that the video sequence can accurately capture high-frequency vibration information, and the entire process does not require physical sensors to contact the measured object, realizing non-contact high-precision vibration measurement.

[0075] Further preferably, step S3 comprises:

[0076] S31, for each to-be-processed resolution frame (low-resolution frame) in the to-be-analyzed video sequence , construct an energy function for describing the reconstruction relationship between the low-resolution image and the target high-resolution image: ; wherein is a downsampling operator, is a reconstruction error term, is a regularization term, is a balance coefficient.

[0077] S32, minimize the energy function by an iterative optimization algorithm to restore a preset resolution frame (high-resolution frame);

[0078] S33, repeat S31-S32 to generate a super-resolution video sequence .

[0079] wherein the to-be-processed resolution frame , is the image size, the preset resolution frame , is the super-resolution multiple.

[0080] As an option, the iterative optimization algorithm adopts gradient descent method, conjugate gradient method or convex optimization algorithm.

[0081] ​​In step S31, the reconstruction relationship between the low-resolution image and the target high-resolution image is described by constructing an energy function. The energy function not only includes a downsampling operator reflecting the image degradation process, but also incorporates a reconstruction error term to measure the degree of image quality improvement, and introduces a regularization term to constrain the solution space to ensure the naturalness and reasonableness of the reconstruction result. The balance coefficient is used to adjust the weight of each term to achieve the optimal reconstruction effect. In step S32, an iterative optimization algorithm such as gradient descent method or convex optimization algorithm is used to minimize the energy function, gradually approaching and restoring the pre-set high-resolution frame. This process continuously adjusts the image pixel value, so that the reconstructed high-resolution frame is visually closer to the real scene. In step S33, by repeatedly executing S31 and S32, each frame in the entire video sequence to be analyzed is super-resolution reconstructed to generate a super-resolution video sequence with higher clarity and detail performance, providing an image basis for subsequent optical flow analysis and vibration feature extraction.

[0082] Further preferably, step S4 comprises:

[0083] S41, calculating the super-resolution video sequence of consecutive frames of optical flow field , obtaining the instantaneous motion speed of each pixel point .

[0084] In step S41, the optical flow field calculation of the video sequence after super-resolution processing can use gradient constraint-based optical flow algorithm (such as Lucas-Kanade algorithm or Farneback algorithm) to analyze pixel motion frame by frame; in actual operation, considering the balance between calculation efficiency and accuracy, the optical flow initial value of the low-resolution layer can be calculated first, and then gradually refined to the high-resolution layer.

[0085] S42, by threshold segmentation, region detection and other methods, determine the target vibration region of the measured object (such as the bearing seat of the rotating equipment or the specific marker point of the vibration table), calculate the average motion speed of all pixel points in the region, including: , is the number of pixels in the target region, is the average motion speed of the region;

[0086] S43, according to the average motion speed and determine the vibration time domain signal .

[0087] Specifically, in step S43, after obtaining the average motion speed of the region, the speed is integrated to calculate the average displacement increment of the region: , This represents the average displacement increment in the horizontal direction. The average displacement increment in the vertical direction is used to calculate the first displacement increment by accumulating the displacement increments. Frame relative to the initial frame ( Cumulative displacement: (Similarly, one is the cumulative displacement in the horizontal direction, and the other is the cumulative displacement in the vertical direction.) Finally, in terms of time... Using the horizontal axis as the x-axis and the cumulative displacement in the horizontal / vertical direction as the y-axis, a vibration time-domain displacement signal is constructed: or .

[0088] More preferably, step S5 includes:

[0089] S51. Transform the time-domain signal using Fast Fourier Transform. Convert to a frequency domain signal, and extract the frequency component with the largest amplitude in the frequency domain signal as the vibration characteristic frequency of the object under test;

[0090] S52. Determine the vibration amplitude A based on the amplitude of the frequency domain signal at the vibration characteristic frequency and the pre-calibrated coefficient k.

[0091] In step S5, the vibration feature extraction process is implemented by first preprocessing the vibration time-domain signal obtained in step S4, including detrending and windowing (such as Hanning window or flat-top window). The preprocessed signal is then converted into a frequency domain signal using a Fast Fourier Transform (FFT). During the spectrum analysis stage, significant peaks in the spectrum are detected first, and noise interference is eliminated by setting a dynamic threshold to ensure accurate identification of the true vibration characteristic frequencies. For cases with multiple similar peaks, a parabolic fitting method is used to improve frequency resolution and accurately determine the dominant frequency position. Regarding amplitude calculation, this invention uses experimental calibration to determine the conversion coefficient k. In practice, calibration is performed using a standard vibration table with known amplitude to establish the correspondence between the frequency domain amplitude and the actual vibration displacement.

[0092] Example 2

[0093] Please combine Figure 3 Based on the same inventive concept, this application proposes a visual vibration measurement system integrating super-resolution and frame interpolation technologies in another specific embodiment, used to implement the visual vibration measurement method as described in Embodiment 1. The system includes a video acquisition module, a frame interpolation processing module, a super-resolution processing module, an optical flow analysis module, and a spectrum analysis module. The video acquisition module is used to acquire the original video sequence of the object under test through a video acquisition device that has no physical connection with the device under test. The original frame rate of the sequence is F; where For the first Frame image, For frame sampling interval, time domain coverage The frame interpolation module is used to process the original video sequence. Frame interpolation is performed to form a video sequence to be analyzed that satisfies the Nyquist sampling theorem after interpolation. The frame rate after sequence interpolation is ;in For the first frame after interpolation Frame image, where N is the number of frames in the original video sequence. This represents the number of frames in the video sequence after frame interpolation. is the index of the original video frame, and m is the index of the video frame after interpolation. The frame interpolation ratio is used; the super-resolution processing module is used to acquire the video sequence to be analyzed. Super-resolution video sequence after super-resolution processing ;in High-resolution frames; the optical flow analysis module is used to extract super-resolution video sequences of the vibration region of the target object. The pixel motion information is converted into vibration time-domain signals. The spectrum analysis module is used to analyze vibration time-domain signals. Determine the vibration characteristic frequency of the object being measured. and amplitude .

[0094] Working Principle: The visual vibration measurement system integrating super-resolution and frame interpolation technologies proposed in this application first acquires the original video sequence of the object under test using a video acquisition device without physical connection through a video acquisition module. Then, a frame interpolation module performs frame interpolation on the original video, generating intermediate frames by calculating the optical flow field and constructing an interpolation model, thereby increasing the video frame rate to meet the Nyquist sampling theorem requirements. Next, a super-resolution processing module performs super-resolution reconstruction on the interpolated video using an energy function and iterative optimization algorithm, enhancing image resolution. An optical flow analysis module extracts pixel motion information from the target vibration region and converts it into a vibration time-domain signal. Finally, a spectrum analysis module performs spectrum analysis on the time-domain signal to determine the vibration characteristic frequencies and amplitudes of the object under test, achieving high-precision vibration measurement.

[0095] For specific limitations regarding the visual vibration measurement system that integrates super-resolution and frame interpolation technologies, please refer to the limitations of the visual vibration measurement method that integrates super-resolution and frame interpolation technologies mentioned above, which will not be repeated here. It should be noted that each module in the above-mentioned visual vibration measurement system corresponds to steps S1 to S5 in implementing the above-mentioned visual vibration measurement method. The instances and application scenarios implemented by multiple modules and their corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above.

[0096] More specifically, the above-mentioned visual vibration measurement system is suitable for various industrial scenarios requiring high-precision vibration measurement, especially in cases where it is difficult to install sensors or is not allowed to interfere with the measured object. For example, in the power, metallurgy and mining industries, non-contact vibration monitoring can be performed on rotating machinery, motors, pumps, reduction boxes and other equipment to promptly detect equipment abnormalities and prevent faults. At the same time, the system can also be used for health monitoring of large building structures (such as high-rise buildings and bridges) and large industrial equipment (such as wind turbines), and through continuous monitoring of vibration indicators, subsequent equipment fault diagnosis can be achieved.

[0097] In yet another embodiment provided by the present application, an electronic device is also provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the visual vibration measurement method as in Embodiment 1 when executing the computer program.

[0098] In yet another embodiment provided by the present application, a computer-readable storage medium is also provided, storing a computer program, and the processor implementing the steps of the visual vibration measurement method as in Embodiment 1 when executing the computer program.

[0099] In order to more clearly illustrate the present application and its advantages, the following will further explain the method provided by the present application in combination with specific implementation cases and related partial drawings.

[0100] The industrial camera is installed at the monitoring point directly in front of the vibration table, and the focal length, aperture and exposure parameters are adjusted to ensure that the captured image is clear and stable. The vibration amplitude of the vibration table is set to 1 mm, and the vibration frequency is set to 40 Hz to simulate typical working conditions. At the same time, the camera video capture frame rate is set to 30 fps, which is less than the vibration table vibration frequency and does not meet the Nyquist sampling theorem for high-frequency vibration component collection. The resolution is adjusted to 512*512 pixels, and the collected video is used to extract vibration information using the optical flow method. The results are shown in Figure 4 and Figure 5 Figure 4 is a time-domain displacement signal curve, the vertical axis reflects the displacement fluctuation, and the horizontal axis corresponds to time; Figure 5 is a frequency-domain spectrum distribution, the vertical axis represents the frequency amplitude, and the horizontal axis is the frequency value. As can be identified from the figure, the extracted characteristic frequency is about 11.5 Hz, and the amplitude is about 0.63 mm. However, the result deviates significantly from the preset parameters of the vibration table. Due to the low video sampling frequency, the Nyquist sampling theorem (the sampling frequency needs to be greater than twice the highest frequency of the signal) is not met, and the vibration frequency set by the vibration table cannot be accurately extracted.

[0101] ​The collected low frame rate video is interpolated (not limited to specific interpolation method, can use interpolation algorithm based on bidirectional optical flow field estimation, interpolation algorithm based on deep learning network, etc.), and the video frame rate is increased from the initial 30fps to 80fps. At this time, the video sampling frequency (80fps) reaches the Nyquist theory critical value of "2 times the highest frequency of the signal". Based on the interpolated video, the optical flow method is used to extract the vibration information, and the results are as follows Figure 6 and Figure 7 presented: Figure 6 is the time domain displacement signal curve, Figure 7 is the frequency domain spectrum distribution. From the frequency domain graph, it can be observed that there is an incomplete wave peak at the 40Hz frequency position, indicating that although the interpolation meets the theoretical sampling condition, due to the Nyquist critical state, there are still problems of insufficient frequency component capture and incomplete feature extraction. At the same time, in the actual monitoring scene, the non-stationary change of light brings video frame-to-frame gray noise interference to the optical flow matching, the multi-order coupling and nonlinear vibration of the vibration table makes the spectrum multi-peak interweave, plus the camera imaging delay, insufficient spatial resolution amplifies the deviation, and these factors together cause the 40Hz characteristic frequency to present an incomplete wave peak.

[0102] Further improve the video frame rate, increase the frame rate from 30fps to 120fps, at this time the video sampling frequency (120fps) reaches 3 times the highest frequency of the signal, satisfying the Nyquist sampling theorem. Based on the interpolated video, the optical flow method is used to extract the vibration information, and the results are as shown in Figure 8 and Figure 9 , Figure 8 is the time domain displacement signal curve, Figure 9 is the frequency domain spectrum distribution. From the graph, it can be observed that there is an obvious wave peak at the 40Hz frequency, which can accurately identify the characteristic frequency, but the amplitude information (about 0.65mm) has a large deviation from the actual value. This is because the video has insufficient original resolution, resulting in errors in the estimation of pixel displacement by the optical flow method. Further processing of the interpolated video through super-resolution algorithm is needed to enhance the image detail resolution and provide more accurate pixel motion trajectory for the optical flow method, so as to obtain accurate vibration amplitude information.

[0103] The video with a frame rate of 120fps after interpolation is processed by 1.5 times super-resolution, and the resolution is increased from 512*512 to 768*768. By enhancing the image details, the optical flow method provides a more accurate pixel motion basis. The super-resolved video is used to extract the vibration information by the optical flow method, and the results are as follows Figure 10 and Figure 11As shown, the characteristic frequency of 40Hz can be accurately identified, and the amplitude is about 0.83mm. Compared with the video without super-resolution processing (amplitude about 0.65mm), the error with the actual set amplitude (1mm) is further reduced. It is shown that by improving the image resolution through super-resolution technology, the extraction accuracy of the vibration amplitude by the optical flow method can be effectively optimized.

[0104] Further improve the video resolution, and perform 2 times super-resolution processing on the interpolated video. The resolution is improved from 512*512 to 1024*1024. The vibration information is extracted from the super-resolution video by using the optical flow method. The results are shown in Figure 12 and Figure 13 As shown, the characteristic frequency of 40Hz can be accurately identified, and the amplitude is about 0.83mm. Compared with the video without super-resolution processing (amplitude about 0.65mm), the error with the actual set amplitude (1mm) is further reduced. It is shown that by improving the image resolution through super-resolution technology, the extraction accuracy of the vibration amplitude by the optical flow method can be effectively optimized.

[0105] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solutions. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0106] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0107] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A visual vibrometry method fusing super-resolution and interpolation techniques, characterized in that, The method comprises the following steps: S1, collect the original video sequence of the measured object , the original frame rate of the sequence is F; wherein is the original frame image of the first frame, the time domain covers , is the frame sampling interval; S2, For the original video sequence The frame rate object is set in the middle to perform frame interpolation processing, forming a video sequence to be analyzed that satisfies the Nyquist sampling theorem after interpolation. The frame rate after sequence interpolation is ;in For the first frame after interpolation Frame image, where N is the number of frames in the original video sequence. This represents the number of frames in the video sequence after frame interpolation. is the index of the original video frame, and m is the index of the video frame after interpolation. The frame interpolation ratio; Step S2 comprises: S21, calculating adjacent frames in the set frame rate object optical flow field The optical flow calculation satisfies the constraint equation: ; wherein is the first frame image in pixel point gray value, is the optical flow displacement of the pixel; S22, constructing the optical flow field After estimating the pixel motion trajectory, a frame interpolation model is constructed to generate the intermediate frame : where is a fusion weight function, ; S23, based on the intermediate frame After the interpolation, the video sequence to be analyzed is obtained as , and the new frame rate is ; S3, acquiring a video sequence to be analyzed super-resolution video sequence after super-resolution processing ; wherein is a high-resolution frame; Step S3 comprises: S31. Video sequence to be analyzed Each resolution frame to be processed in Constructing the energy function Used to describe the reconstruction relationship between a low-resolution image and a target high-resolution image: ;in For downsampling operators, To reconstruct the error term, For regularization terms, This is the balance coefficient; S32, minimize the energy function by an iterative optimization algorithm , restore the preset resolution frame ; S33, repeat performing S31-S32 to generate the super-resolution video sequence ; wherein the resolution frame to be processed , image size, preset resolution frame , is the super-resolution multiple; S4, extracting pixel motion information of a target vibration region of the measured object to convert into a vibration time domain signal ;​ S5. Based on the vibration time-domain signal Determine the vibration characteristic frequency of the object being measured. and amplitude .

2. The visual vibration measurement method of claim 1, wherein, The iterative optimization algorithm adopts a gradient descent method, a conjugate gradient method or a convex optimization algorithm.

3. The visual vibrometer method of claim 1, wherein, Step S4 comprises: S41, calculating the super-resolution video sequence of consecutive frames of the optical flow field , obtaining the instantaneous motion speed of each pixel point ;​ S42, determine the target vibration region of the measured object, calculate the average motion speed of all pixel points in the region, including: , is the number of target region pixels, is the average motion speed of the region; S43, average motion speed over time series and determining a vibration time domain signal .

4. The visual vibrometer method of claim 1, wherein, Step S5 comprises: S51, converting the time domain signal into a frequency domain signal through fast Fourier transform, and extracting a frequency component with the largest amplitude in the frequency domain signal as a vibration characteristic frequency of the measured object; S52, determining the vibration amplitude A according to the amplitude of the frequency domain signal at the vibration characteristic frequency and in combination with a pre-calibrated coefficient k.

5. The visual vibrometry method of claim 1, wherein, The original video sequence of the measured object is acquired by a video acquisition device without physical connection with the measured device, so as to realize non-contact image acquisition.

6. A visual vibrometry system fusing super-resolution and interpolation techniques for implementing the visual vibrometry method of any one of claims 1-5, characterized in that, The system comprises: The video acquisition module is configured to acquire the original video sequence of the measured object through a video acquisition device without physical connection with the measured device , and a sequence original frame rate is F; wherein is the frame image, is a frame sampling interval, and a time domain covers ; The frame interpolation module is used to process the original video sequence. The frame rate object is set in the middle to perform frame interpolation processing, forming a video sequence to be analyzed that satisfies the Nyquist sampling theorem after interpolation. The frame rate after sequence interpolation is ;in For the first frame after interpolation Frame image, where N is the number of frames in the original video sequence. This represents the number of frames in the video sequence after frame interpolation. is the index of the original video frame, and m is the index of the video frame after interpolation. The frame interpolation ratio; a super-resolution processing module configured to obtain a video sequence to be analyzed a super-resolution video sequence after super-resolution processing ; wherein is a high-resolution frame The optical flow analysis module is used to extract super-resolution video sequences of the vibration region of the target object. The pixel motion information is converted into vibration time-domain signals. ; a spectrum analysis module configured to determine a vibration characteristic frequency and amplitude of the measured object based on the vibration time-domain signal .

7. An electronic device, comprising: The system comprises a memory and a processor, the memory stores a computer program, and the processor realizes the steps of the visual vibration measurement method in any one of claims 1-5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The memory stores a computer program, and the computer program realizes the steps of the visual vibration measurement method in any one of claims 1-5 when executed by the processor.

Images