Visual vibration measurement method, system and equipment integrating super-resolution and frame insertion technologies

Abstract

The invention belongs to the technical field of image processing, and particularly relates to a visual vibration measurement method, system and device fusing super-resolution and frame insertion technology, and the method comprises the steps: collecting a detected object video through an industrial camera in a non-contact manner, improving the video frame rate through the frame insertion technology to meet the Nyquist sampling theorem, and solving the problem of high-frequency vibration information loss; the super-resolution technology is adopted to enhance the image resolution, the non-uniform illumination and background noise interference are overcome, and the infinitesimal displacement detection precision is improved. The system comprises a video acquisition module, a frame insertion processing module, a super-resolution processing module, an optical flow analysis module and a spectrum analysis module, and realizes accurate extraction of vibration characteristic frequency and amplitude. The method does not need additional sensors, reduces the hardware cost, and is suitable for equipment monitoring under various complex working conditions.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a visual vibration measurement method, system, and device that integrates super-resolution and frame interpolation technologies. Background Technology

[0002] Currently, vibration measurement of industrial equipment is mainly divided into two categories: contact and non-contact methods. Contact measurement typically uses accelerometers, strain gauges, etc., directly mounted on the surface of the equipment under test, and collects vibration data through electrical signals. This method is technically mature, but it requires physical contact, which may introduce additional mass loads and affect the vibration characteristics of the object under test. Non-contact measurement technologies include laser Doppler vibration meters and high-speed camera vision measurement. Although they avoid contact issues, they have high requirements for the equipment installation environment and are expensive.

[0003] The existing vibration measurement technology has the following main problems: (1) Contact measurement will change the vibration characteristics of the object being measured, especially for lightweight structures or precision equipment, the measurement error is significant; (2) Non-contact methods (such as laser vibration measurement) are expensive and have strict requirements for the testing environment, making them difficult to be widely used in complex industrial scenarios; (3) Traditional visual vibration measurement methods are limited by camera frame rate and resolution, making it difficult to meet the requirements for accurate measurement of high-frequency vibrations, and are easily affected by factors such as changes in lighting and background noise. These problems seriously restrict the application effect and promotion value of vibration measurement technology in the field of industrial monitoring. Summary of the Invention

[0004] The purpose of this invention is to provide a visual vibration measurement method, system, and device that integrates super-resolution and frame interpolation technologies, so as to overcome the problems of low vibration measurement accuracy and poor anti-interference ability caused by insufficient video frame rate and resolution in existing visual vibration measurement technologies without contacting the device under test.

[0005] The present invention achieves the above objectives through the following technical solutions: Firstly, the present invention proposes a visual vibration measurement method that integrates super-resolution and frame interpolation techniques, the method comprising the following steps: S1. Acquire the original video sequence of the object under test. The original frame rate of the sequence is F; where For the first Frame image, temporal domain coverage , The frame sampling interval; S2, For the original video sequence The frame rate object is set in the middle to perform frame interpolation processing, forming a video sequence to be analyzed that satisfies the Nyquist sampling theorem after interpolation. The frame rate after sequence interpolation is ;in For the first frame after interpolation Frame image, where N is the number of frames in the original video sequence. This represents the number of frames in the video sequence after frame interpolation. is the index of the original video frame, and m is the index of the video frame after interpolation. The frame interpolation ratio; S3. Obtain the video sequence to be analyzed. Super-resolution video sequence after super-resolution processing ;in High-resolution frames; S4. Extract the super-resolution video sequence of the vibration region of the target object. The pixel motion information is converted into vibration time-domain signals. ; S5. Based on the vibration time-domain signal Determine the vibration characteristic frequency of the object being measured. and amplitude .

[0006] Furthermore, step S2 includes: S21. Calculate adjacent frames in the set frame rate object. Optical flow field The optical flow calculation satisfies the constraint equations: ;in For the first Frame image in The grayscale value of a pixel. This represents the optical flow displacement of the pixel; S22, through the optical flow field After estimating the pixel motion trajectory, an interpolation model is constructed to generate intermediate frames. : ,in To integrate the weighting function, ; S23, Based on intermediate frames After frame interpolation, the resulting video sequence to be analyzed is: The new frame rate is .

[0007] Furthermore, step S3 includes: S31. Video sequence to be analyzed Each resolution frame to be processed in Constructing the energy function Used to describe the reconstruction relationship between a low-resolution image and a target high-resolution image: ;in For downsampling operators, To reconstruct the error term, For regularization terms, This is the balance coefficient; S32. Minimize the energy function using an iterative optimization algorithm. Recover the preset resolution frame ; S33. Repeat steps S31-S32 to generate a super-resolution video sequence. ; Among them, the frames to be processed are , Image size, preset resolution frame , This is the super-resolution magnification.

[0008] Furthermore, the iterative optimization algorithm employs gradient descent, conjugate gradient, or convex optimization algorithms.

[0009] Furthermore, step S4 includes: S41. Calculate super-resolution video sequences Medium continuous frames Optical flow field Get each pixel instantaneous velocity ; S42. Determine the vibration region of the target object and calculate the average velocity of all pixels within that region, including: , It is the number of pixels in the target area. The average speed of movement in the region; S43. Based on the average velocity over the time series and Determine the vibration time-domain signal .

[0010] Furthermore, step S5 includes: S51. Convert the time-domain signal into a frequency-domain signal using fast Fourier transform, and extract the frequency component with the largest amplitude in the frequency-domain signal as the vibration characteristic frequency of the object under test. S52. Determine the vibration amplitude A based on the amplitude of the frequency domain signal at the vibration characteristic frequency, combined with the pre-calibrated coefficient k.

[0011] Furthermore, the original video sequence of the object under test is acquired through a video acquisition device that has no physical connection with the device under test, so as to achieve non-contact image acquisition.

[0012] Secondly, the present invention proposes a visual vibration measurement system that integrates super-resolution and frame interpolation techniques to implement the visual vibration measurement method described above. The system includes: The video acquisition module is used to acquire the original video sequence of the object under test through a video acquisition device that has no physical connection with the device under test. The original frame rate of the sequence is F; where For the first Frame image, For frame sampling interval, time domain coverage ; The frame interpolation module is used to process the original video sequence. The frame rate object is set in the middle to perform frame interpolation processing, forming a video sequence to be analyzed that satisfies the Nyquist sampling theorem after interpolation. The frame rate after sequence interpolation is ;in For the first frame after interpolation Frame image, where N is the number of frames in the original video sequence. This represents the number of frames in the video sequence after frame interpolation. is the index of the original video frame, and m is the index of the video frame after interpolation. The frame interpolation ratio; The super-resolution processing module is used to acquire the video sequence to be analyzed. Super-resolution video sequence after super-resolution processing ;in High-resolution frames; The optical flow analysis module is used to extract super-resolution video sequences of the vibration region of the target object. The pixel motion information is converted into vibration time-domain signals. ; The spectrum analysis module is used to analyze the vibration time-domain signal. Determine the vibration characteristic frequency of the object being measured. and amplitude .

[0013] Thirdly, the present invention proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the visual vibration measurement method described above.

[0014] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the visual vibration measurement method described above.

[0015] The beneficial effects of this invention are as follows: This invention effectively overcomes the limitations of traditional measurement methods through non-contact vibration measurement. Technically, it utilizes frame interpolation to increase the video frame rate, ensuring the sampling process satisfies the Nyquist theorem and resolving the loss of high-frequency vibration information caused by low-frequency sampling. Simultaneously, it employs super-resolution technology to enhance image resolution, significantly improving the detection accuracy of minute displacements and providing a more reliable image foundation for subsequent vibration analysis. In terms of application, this method eliminates reliance on physical sensors and avoids the mass load effect common in contact measurements, making it suitable for equipment monitoring under various complex operating conditions. Compared to existing technologies, this invention significantly reduces hardware costs and environmental requirements; ordinary industrial cameras can meet the measurement needs, while effectively suppressing background noise interference. Attached Figure Description

[0016] Figure 1 A flowchart illustrating a visual vibration measurement method that integrates super-resolution and frame interpolation techniques according to an embodiment of this application; Figure 2 This is another flowchart illustrating a visual vibration measurement method that integrates super-resolution and frame interpolation techniques, as provided in one embodiment of this application. Figure 3 A system block diagram of a visual vibration measurement system that integrates super-resolution and frame interpolation technologies according to an embodiment of this application; Figure 4 This is a time-domain displacement signal curve when the original frame rate of video acquisition is 30fps in the experimental case of the specific implementation of this application; Figure 5 This is a time-frequency domain spectrum distribution diagram of the video acquisition with an original frame rate of 30fps in the experimental case of a specific implementation of this application; Figure 6 This is a time-domain displacement signal curve when the video frame is interpolated to 80fps in an experimental case of a specific implementation of this application; Figure 7 This is a time-frequency domain spectrum distribution diagram of video frame interpolation to 80fps in an experimental case of a specific implementation of this application; Figure 8 This is a time-domain displacement signal curve when the video frame is interpolated to 120fps in an experimental case of a specific implementation of this application; Figure 9 This is a time-frequency domain spectrum distribution diagram of video frame interpolation to 120fps in an experimental case of a specific implementation of this application; Figure 10 The time-domain shift signal curve of the video interpolated to 120fps in the specific implementation example of this application is shown in the experimental case. Figure 11 The frequency domain spectrum distribution of the video with frame interpolation to 120fps in the experimental case of the specific implementation of this application is shown in the figure. Figure 12 This is a time-domain shift signal curve of a video interpolated to 120fps using 2x super-resolution processing in an experimental case of a specific implementation of this application. Figure 13 The frequency domain spectrum distribution of the video with frame interpolation to 120fps in the experimental case of the specific implementation of this application is shown in the figure. Detailed Implementation

[0017] The present application will now be described in further detail with reference to the accompanying drawings. It should be noted that the following specific embodiments are only used to further illustrate the present application and should not be construed as limiting the scope of protection of the present application. Those skilled in the art can make some non-essential improvements and adjustments to the present application based on the above application content.

[0018] Example 1 Please combine Figure 1 and Figure 2 A specific embodiment of this application proposes a visual vibration measurement method that integrates super-resolution and frame interpolation techniques. The method includes the following steps: S1. Acquire the original video sequence of the object under test. The original frame rate of the sequence is F; where For the first Frame image, temporal domain coverage , The frame sampling interval is used to obtain the original video sequence of the object under test through a video acquisition device that has no physical connection with the object under test (such as industrial equipment that needs to be diagnosed) to achieve non-contact image acquisition.

[0019] S2, For the original video sequence The frame rate object is set in the middle to perform frame interpolation processing, forming a video sequence to be analyzed that satisfies the Nyquist sampling theorem after interpolation. The frame rate after sequence interpolation is ;in For the first frame after interpolation Frame image, where N is the number of frames in the original video sequence. This represents the number of frames in the video sequence after frame interpolation. is the index of the original video frame, and m is the index of the video frame after interpolation. This represents the frame interpolation ratio.

[0020] Traditional visual vibration measurement methods often suffer from insufficient camera frame rates, leading to the loss of high-frequency vibration information and failing to meet the Nyquist sampling theorem (the sampling frequency must be greater than twice the highest frequency of the signal). Therefore, this application employs frame interpolation technology. By calculating the optical flow field between adjacent frames, the pixel motion trajectory is estimated, and an interpolation model is constructed to generate intermediate frames, thereby increasing the video frame rate. This process ensures that the interpolated video sequence meets the Nyquist sampling theorem requirements, accurately captures high-frequency vibration information, and avoids information loss.

[0021] Due to the interpolated video Due to physical limitations of imaging equipment and transmission losses, there is a problem of insufficient resolution. Super-resolution can recover high-resolution details by learning complementary information between low-resolution frames.

[0022] S3. Obtain the video sequence to be analyzed. Super-resolution video sequence after super-resolution processing ;in It is a high-resolution frame.

[0023] S4. Extract the super-resolution video sequence of the vibration region of the target object. The pixel motion information is converted into vibration time-domain signals. .

[0024] S5. Based on the vibration time domain signal Determine the vibration characteristic frequency of the object being measured. and amplitude .

[0025] In this application, the visual vibration measurement method eliminates the need to install vibration sensors on the surface of the object under test. Vibration measurement is achieved solely through a standard industrial camera, making it applicable to scenarios where sensor installation is difficult or where interference with the object under test is not permitted. Furthermore, the original video of the object under test in this application can be video information from a vibration table, or video information from various production equipment such as motors, pumps, and gearboxes.

[0026] Understandably, this application proposes combining frame interpolation with super-resolution processing. Frame interpolation ensures that the video sampling rate meets the Nyquist requirements, effectively avoiding the loss of high-frequency vibration information; while super-resolution significantly improves image detail resolution, providing a more accurate basis for pixel motion analysis. Secondly, this method establishes a complete "video acquisition - frame rate enhancement - resolution enhancement - vibration extraction" technology chain, completely eliminating the mass load effect caused by sensor installation through non-contact measurement, while reducing the performance requirements of hardware devices.

[0027] More preferably, step S2 includes: S21. Calculate adjacent frames in the set frame rate object. Optical flow field The optical flow calculation satisfies the constraint equations: ;in For the first Frame image in The grayscale value of a pixel. This represents the optical flow displacement of the pixel.

[0028] S22, through the optical flow field After estimating the pixel motion trajectory, an interpolation model is constructed to generate intermediate frames. : ,in To integrate the weighting function, .

[0029] S23, Based on intermediate frames After frame interpolation, the resulting video sequence to be analyzed is: The new frame rate is .

[0030] It should be noted that, in this application, "the object for setting the frame rate" refers to a specific video segment or the entire video sequence that is selected for frame interpolation to improve the frame rate in the original video sequence.

[0031] Specifically, in visual vibration measurement methods, high-frequency vibration information is lost because the frame rate of the original video sequence does not meet the requirements of the Nyquist sampling theorem. Therefore, it is necessary to perform frame interpolation on the video sequence.

[0032] In step S21, the optical flow field of adjacent frames in the set frame rate object is calculated using an algorithm. During this process, the grayscale value and optical flow displacement of each pixel are accurately captured to satisfy the constraint equations of optical flow calculation, ensuring the accuracy of motion estimation. Subsequently, in step S22, the system uses the optical flow field to estimate the motion trajectory of the pixels and intelligently generates intermediate frames through the constructed frame interpolation model. In this process, the application of the fusion weight function makes the generation of intermediate frames smoother and more consistent with actual motion laws, effectively improving the frame interpolation quality. Finally, in step S23, frame interpolation processing is performed based on the generated intermediate frames to obtain a new video sequence that meets the requirements of the Nyquist sampling theorem. The increase in the new frame rate ensures that the video sequence can accurately capture high-frequency vibration information. The entire process does not require physical sensors to contact the object being measured, realizing non-contact high-precision vibration measurement.

[0033] More preferably, step S3 includes: S31. Video sequence to be analyzed Each resolution frame to be processed in (Low-resolution frame), construct energy function Used to describe the reconstruction relationship between a low-resolution image and a target high-resolution image: ;in For downsampling operators, To reconstruct the error term, For regularization terms, This is the balance coefficient.

[0034] S32. Minimize the energy function using an iterative optimization algorithm. Recover the preset resolution frame (High-resolution frame); S33. Repeat steps S31-S32 to generate a super-resolution video sequence. .

[0035] Among them, the frames to be processed are , Image size, preset resolution frame , This is the super-resolution magnification.

[0036] Alternatively, the iterative optimization algorithm may employ gradient descent, conjugate gradient, or convex optimization.

[0037] In step S31, this application constructs an energy function to characterize the reconstruction relationship between the low-resolution image and the target high-resolution image. This energy function not only includes a downsampling operator reflecting the image degradation process but also incorporates a reconstruction error term to measure the degree of image quality improvement. A regularization term is introduced to constrain the solution space, ensuring the naturalness and rationality of the reconstruction result. A balance coefficient is used to adjust the weights of each term to achieve the optimal reconstruction effect. In step S32, an iterative optimization algorithm, such as gradient descent or convex optimization, is used to minimize the energy function, gradually approximating and recovering the preset high-resolution frame. This process continuously adjusts the image pixel values, making the reconstructed high-resolution frame visually closer to the real scene. In step S33, by repeatedly executing S31 and S32, super-resolution reconstruction is performed on each frame of the entire video sequence to be analyzed, generating a super-resolution video sequence with higher clarity and detail, providing an image foundation for subsequent optical flow analysis and vibration feature extraction.

[0038] More preferably, step S4 includes: S41. Calculate super-resolution video sequences Medium continuous frames Optical flow field Get each pixel instantaneous velocity .

[0039] In step S41, optical flow field calculation is performed on the video sequence after super-resolution processing. Gradient-constrained optical flow algorithms (such as the Lucas-Kanade algorithm or the Farneback algorithm) can be used to analyze pixel motion frame by frame. In actual operation, considering the balance between computational efficiency and accuracy, the initial value of optical flow of the low-resolution layer can be calculated first, and then gradually refined to the high-resolution layer.

[0040] S42. Determine the target vibration region of the object under test (such as the bearing seat of a rotating device or a specific marker point of a vibration table) through threshold segmentation, region detection, etc., and calculate the average motion velocity of all pixels in this region, including: , It is the number of pixels in the target area. The average speed of movement in the region; S43. Based on the average velocity over the time series and Determine the vibration time-domain signal .

[0041] Specifically, in step S43, after obtaining the average velocity of the region, the velocity is integrated to calculate the average displacement increment of the region: , This represents the average displacement increment in the horizontal direction. The average displacement increment in the vertical direction is used to calculate the first displacement increment by accumulating the displacement increments. Frame relative to the initial frame ( Cumulative displacement: (Similarly, one is the cumulative displacement in the horizontal direction, and the other is the cumulative displacement in the vertical direction.) Finally, in terms of time... Using the horizontal axis as the x-axis and the cumulative displacement in the horizontal / vertical direction as the y-axis, a vibration time-domain displacement signal is constructed: or .

[0042] More preferably, step S5 includes: S51. Transform the time-domain signal using Fast Fourier Transform. Convert to a frequency domain signal, and extract the frequency component with the largest amplitude in the frequency domain signal as the vibration characteristic frequency of the measured object; S52. Determine the vibration amplitude A based on the amplitude of the frequency domain signal at the vibration characteristic frequency and the pre-calibrated coefficient k.

[0043] In step S5, the vibration feature extraction process is implemented by first preprocessing the vibration time-domain signal obtained in step S4, including detrending and windowing (such as Hanning window or flat-top window). The preprocessed signal is then converted into a frequency domain signal using a Fast Fourier Transform (FFT). During the spectrum analysis stage, significant peaks in the spectrum are detected first, and noise interference is eliminated by setting a dynamic threshold to ensure accurate identification of the true vibration characteristic frequencies. For cases with multiple similar peaks, a parabolic fitting method is used to improve frequency resolution and accurately determine the dominant frequency position. Regarding amplitude calculation, this invention uses experimental calibration to determine the conversion coefficient k. In practice, calibration is performed using a standard vibration table with known amplitude to establish the correspondence between the frequency domain amplitude and the actual vibration displacement.

[0044] Example 2 Please combine Figure 3 Based on the same inventive concept, this application proposes a visual vibration measurement system integrating super-resolution and frame interpolation technologies in another specific embodiment, used to implement the visual vibration measurement method as described in Embodiment 1. The system includes a video acquisition module, a frame interpolation processing module, a super-resolution processing module, an optical flow analysis module, and a spectrum analysis module. The video acquisition module is used to acquire the original video sequence of the object under test through a video acquisition device that has no physical connection with the device under test. The original frame rate of the sequence is F; where For the first Frame image, For frame sampling interval, time domain coverage The frame interpolation module is used to process the original video sequence. Frame interpolation is performed to form a video sequence to be analyzed that satisfies the Nyquist sampling theorem after interpolation. The frame rate after sequence interpolation is ;in For the first frame after interpolation Frame image, where N is the number of frames in the original video sequence. This represents the number of frames in the video sequence after frame interpolation. is the index of the original video frame, and m is the index of the video frame after interpolation. The frame interpolation ratio is used; the super-resolution processing module is used to acquire the video sequence to be analyzed. Super-resolution video sequence after super-resolution processing ;in High-resolution frames; the optical flow analysis module is used to extract super-resolution video sequences of the vibration region of the target object. The pixel motion information is converted into vibration time-domain signals. The spectrum analysis module is used to analyze vibration time-domain signals. Determine the vibration characteristic frequency of the object being measured. and amplitude .

[0045] Working Principle: The visual vibration measurement system integrating super-resolution and frame interpolation technologies proposed in this application first acquires the original video sequence of the object under test using a video acquisition device without physical connection through a video acquisition module. Then, a frame interpolation module performs frame interpolation on the original video, generating intermediate frames by calculating the optical flow field and constructing an interpolation model, thereby increasing the video frame rate to meet the Nyquist sampling theorem requirements. Next, a super-resolution processing module performs super-resolution reconstruction on the interpolated video using an energy function and iterative optimization algorithm, enhancing image resolution. An optical flow analysis module extracts pixel motion information from the target vibration region and converts it into a vibration time-domain signal. Finally, a spectrum analysis module performs spectrum analysis on the time-domain signal to determine the vibration characteristic frequencies and amplitudes of the object under test, achieving high-precision vibration measurement.

[0046] For specific limitations regarding the visual vibration measurement system that integrates super-resolution and frame interpolation technologies, please refer to the limitations of the visual vibration measurement method that integrates super-resolution and frame interpolation technologies mentioned above, which will not be repeated here. It should be noted that each module in the above-mentioned visual vibration measurement system corresponds to steps S1 to S5 in implementing the above-mentioned visual vibration measurement method. The instances and application scenarios implemented by multiple modules and their corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above.

[0047] More specifically, the aforementioned visual vibration measurement system is suitable for various industrial scenarios requiring high-precision vibration measurement, especially in situations where it is difficult to install sensors or where interference with the measured object is not permitted. For example, in industries such as power, metallurgy, and mining, it can perform non-contact vibration monitoring of equipment such as rotating machinery, motors, pumps, and gearboxes, promptly detecting equipment anomalies and preventing failures. Simultaneously, the system can also be used for health monitoring of large building structures (such as high-rise buildings and bridges) and large industrial equipment (such as wind turbines), enabling subsequent equipment fault diagnosis through continuous monitoring of vibration indicators.

[0048] In another embodiment of the present invention, an electronic device is also provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the visual vibration measurement method as described in Embodiment 1.

[0049] In another embodiment of the present invention, a computer-readable storage medium is also provided, storing a computer program that, when executed by a processor, implements the steps of the visual vibration measurement method as described in Embodiment 1.

[0050] To more clearly illustrate the present invention and its advantages, the method provided by the present invention will be further explained below in conjunction with specific implementation examples and related partial figures.

[0051] An industrial camera was mounted directly in front of the vibration table at the monitoring point. The focus, aperture, and exposure parameters were adjusted to ensure clear and stable image capture. The vibration table was set to a vibration amplitude of 1mm and a vibration frequency of 40Hz to simulate typical working conditions. The camera's video capture frame rate was set to 30fps, which is lower than the vibration table's frequency and does not meet the Nyquist sampling theorem's requirement for high-frequency vibration components. The resolution was adjusted to 512*512 pixels. Optical flow was used to extract vibration information from the acquired video. The results are as follows: Figure 4 and Figure 5 As shown, Figure 4 This is a time-domain displacement signal curve, with the vertical axis reflecting displacement fluctuations and the horizontal axis corresponding to time. Figure 5 The graph shows the frequency spectrum distribution, with the vertical axis representing frequency amplitude and the horizontal axis representing frequency value. The extracted characteristic frequency is approximately 11.5 Hz, and the amplitude is approximately 0.63 mm. However, this result deviates significantly from the preset parameters of the vibration table because the video sampling frequency is too low, failing to satisfy the Nyquist sampling theorem (the sampling frequency must be greater than twice the highest frequency of the signal), thus making it impossible to accurately extract the vibration frequency set by the vibration table.

[0052] Frame interpolation was performed on the acquired low-frame-rate video (any specific interpolation method could be used, such as interpolation algorithms based on bidirectional optical flow estimation or deep learning networks), increasing the video frame rate from the initial 30fps to 80fps. At this point, the video sampling frequency (80fps) reached the Nyquist theorem critical value of "twice the highest signal frequency". Based on the interpolated video, vibration information was extracted using optical flow, and the results are as follows. Figure 6 and Figure 7 Presentation: Figure 6 This is the time-domain displacement signal curve. Figure 7 The frequency domain spectrum distribution shows an incomplete peak at 40Hz. This indicates that although the theoretical sampling conditions are met by frame interpolation, there are still problems with insufficient capture of frequency components and incomplete feature extraction due to the Nyquist critical state. In the actual monitoring scenario, non-stationary changes in light cause grayscale noise interference between video frames, which interferes with optical flow matching. The multi-level coupling and nonlinear vibration of the shaking table cause the spectrum to intertwine with multiple peaks. In addition, camera imaging delay, insufficient spatial resolution and amplification deviation all contribute to the incomplete peak at the 40Hz characteristic frequency.

[0053] Further increasing the video frame rate from 30fps to 120fps, the video sampling frequency (120fps) reaches three times the highest signal frequency, satisfying the Nyquist sampling theorem. Based on the interpolated video, vibration information is extracted using optical flow, as shown in the results. Figure 8 and Figure 9 As shown, Figure 8 This is the time-domain displacement signal curve. Figure 9The frequency spectrum distribution is shown in the figure. A distinct peak appears at 40Hz, allowing for accurate identification of the characteristic frequency. However, the amplitude information (approximately 0.65mm) deviates significantly from the actual value. This is due to insufficient original video resolution, leading to errors in the estimation of pixel displacement by the optical flow method. Further processing using a super-resolution algorithm is needed to enhance the image detail resolution and provide more accurate pixel motion trajectories for the optical flow method, thereby obtaining accurate vibration amplitude information.

[0054] A 1.5x super-resolution process was performed on the interpolated video, increasing the resolution from 512*512 to 768*768. This enhanced image detail provided a more accurate basis for pixel motion analysis using optical flow. Vibration information was then extracted from the super-resolution video using optical flow, with the following results: Figure 10 and Figure 11 As shown, the characteristic frequency of 40Hz and the amplitude of approximately 0.83mm can be accurately identified. Compared with the video without super-resolution processing (amplitude of approximately 0.65mm), the error with the actual set amplitude (1mm) is further reduced. This indicates that improving image resolution through super-resolution technology can effectively optimize the extraction accuracy of vibration amplitude by the optical flow method.

[0055] To further enhance the video resolution, the interpolated video underwent a 2x super-resolution process, increasing the resolution from 512*512 to 1024*1024. Optical flow was then used to extract vibration information from the super-resolution video, with the following results: Figure 12 and Figure 13 As shown, the characteristic frequency can be accurately identified as 40Hz, with an amplitude of approximately 0.92mm. Compared to the video processed by 1.5 super-resolution (amplitude of approximately 0.83mm), the error with the actual set amplitude (1mm) is further reduced, indicating that higher super-resolution can continuously optimize the extraction effect of vibration amplitude by optical flow method by enhancing image details.

[0056] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0057] In addition, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0058] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A visual vibration measurement method integrating super-resolution and frame interpolation techniques, characterized in that, The method includes the following steps: S1. Acquire the original video sequence of the object under test. The original frame rate of the sequence is F; where For the first Frame image, temporal domain coverage , The frame sampling interval; S2, For the original video sequence The frame rate object is set in the middle to perform frame interpolation processing, forming a video sequence to be analyzed that satisfies the Nyquist sampling theorem after interpolation. The frame rate after sequence interpolation is ;in For the first frame after interpolation Frame image, where N is the number of frames in the original video sequence. This represents the number of frames in the video sequence after frame interpolation. is the index of the original video frame, and m is the index of the video frame after interpolation. The frame interpolation ratio; S3. Obtain the video sequence to be analyzed. Super-resolution video sequence after super-resolution processing ;in High-resolution frames; S4. Extract the super-resolution video sequence of the vibration region of the target object under test. The pixel motion information is converted into vibration time-domain signals. ; S5. Based on the vibration time-domain signal Determine the vibration characteristic frequency of the object being measured. and amplitude .

2. The visual vibration measurement method integrating super-resolution and frame interpolation techniques according to claim 1, characterized in that, Step S2 includes: S21. Calculate adjacent frames in the set frame rate object. Optical flow field The optical flow calculation satisfies the constraint equations: ;in For the first Frame image in The grayscale value of a pixel. This represents the optical flow displacement of the pixel; S22, through the optical flow field After estimating the pixel motion trajectory, an interpolation model is constructed to generate intermediate frames. : ,in To integrate the weighting function, ; S23, Based on intermediate frames After frame interpolation, the resulting video sequence to be analyzed is: The new frame rate is .

3. The visual vibration measurement method integrating super-resolution and frame interpolation techniques according to claim 1, characterized in that, Step S3 includes: S31. Video sequence to be analyzed Each resolution frame to be processed in Constructing the energy function Used to describe the reconstruction relationship between a low-resolution image and a target high-resolution image: ;in For downsampling operators, To reconstruct the error term, For regularization terms, This is the balance coefficient; S32. Minimize the energy function using an iterative optimization algorithm. Recover the preset resolution frame ; S33. Repeat steps S31-S32 to generate a super-resolution video sequence. ; Among them, the frames to be processed are , Image size, preset resolution frame , This is the super-resolution magnification.

4. The visual vibration measurement method integrating super-resolution and frame interpolation techniques according to claim 3, characterized in that, The iterative optimization algorithm employs gradient descent, conjugate gradient, or convex optimization algorithms.

5. The visual vibration measurement method integrating super-resolution and frame interpolation techniques according to claim 1, characterized in that, Step S4 includes: S41. Calculate super-resolution video sequences Medium continuous frames Optical flow field Get each pixel instantaneous velocity ; S42. Determine the vibration region of the target object and calculate the average velocity of all pixels within that region, including: , It is the number of pixels in the target area. The average speed of movement in the region; S43. Based on the average velocity over the time series and Determine the vibration time-domain signal .

6. The visual vibration measurement method integrating super-resolution and frame interpolation techniques according to claim 1, characterized in that, Step S5 includes: S51. Convert the time-domain signal into a frequency-domain signal using fast Fourier transform, and extract the frequency component with the largest amplitude in the frequency-domain signal as the vibration characteristic frequency of the object under test. S52. Determine the vibration amplitude A based on the amplitude of the frequency domain signal at the vibration characteristic frequency, combined with the pre-calibrated coefficient k.

7. The visual vibration measurement method integrating super-resolution and frame interpolation techniques according to claim 1, characterized in that, The original video sequence of the object under test is acquired through a video acquisition device that has no physical connection with the device under test, so as to achieve non-contact image acquisition.

8. A visual vibration measurement system integrating super-resolution and frame interpolation technologies, used to implement the visual vibration measurement method as described in any one of claims 1-7, characterized in that, The system includes: The video acquisition module is used to acquire the original video sequence of the object under test through a video acquisition device that has no physical connection with the device under test. The original frame rate of the sequence is F; where For the first Frame image, For frame sampling interval, time domain coverage ; The frame interpolation module is used to process the original video sequence. The frame rate object is set in the middle to perform frame interpolation processing, forming a video sequence to be analyzed that satisfies the Nyquist sampling theorem after interpolation. The frame rate after sequence interpolation is ;in For the first frame after interpolation Frame image, where N is the number of frames in the original video sequence. This represents the number of frames in the video sequence after frame interpolation. is the index of the original video frame, and m is the index of the video frame after interpolation. The frame interpolation ratio; The super-resolution processing module is used to acquire the video sequence to be analyzed. Super-resolution video sequence after super-resolution processing ;in High-resolution frames; The optical flow analysis module is used to extract super-resolution video sequences of the vibration region of the target object. The pixel motion information is converted into vibration time-domain signals. ; The spectrum analysis module is used to analyze the vibration time-domain signal. Determine the vibration characteristic frequency of the object being measured. and amplitude .

9. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the visual vibration measurement method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the steps of the visual vibration measurement method according to any one of claims 1-7.

Images