Mobile vibration inspection system and method carried on mechanical dog

Abstract

The invention discloses a mobile vibration inspection system and method carried on a mechanical dog, and relates to intelligent detection. The method comprises the following steps: preprocessing video data to obtain preprocessed video data; performing complex controllable pyramid decomposition on the preprocessed video data to obtain multi-scale and multi-direction amplitudes and phase differences of each frame of video; according to the phase difference, adopting a phase optical flow method to extract vibration displacement; performing Fourier transform frequency analysis on the vibration displacement to obtain vibration frequency characteristics; and according to the obtained amplitude and phase difference and the obtained vibration frequency characteristics, amplifying and reconstructing the original video to obtain a reconstructed video after motion amplification. Aiming at the poor reconstruction stability of the motion video, the method achieves the video stabilization processing through an ORB algorithm and affine transformation, carries out the multi-scale decomposition through a complex controllable pyramid, employs a phase optical flow method to extract the tiny vibration, achieves the amplification of the tiny vibration while maintaining the quality of the video, and improves the reconstruction quality of the video.

Description

Technical Field

[0001] This application relates to the field of intelligent detection, and more specifically, to a mobile vibration inspection system and method mounted on a mechanical dog. Background Technology

[0002] In industries such as manufacturing, energy, transportation, and healthcare, vibration detection of equipment components is a core technical means to ensure safe production, improve operational efficiency, and optimize equipment maintenance. Especially in complex environments such as high-temperature, high-risk, and dusty conditions, abnormal vibration of critical equipment components such as motor bearings, gearboxes, and rotating machinery can trigger a chain reaction of failures, and in severe cases, even lead to major safety accidents such as explosions and fires. Therefore, accurate and timely detection and analysis of the vibration characteristics of these equipment components are of great significance for preventing failures and ensuring production safety.

[0003] Traditional vibration detection primarily relies on contact sensors such as accelerometers and displacement gauges for data acquisition. While these methods offer high accuracy in certain scenarios, they suffer from significant limitations: First, the sensors must be directly mounted on the surface of the object being measured, which poses installation difficulties and safety hazards for components subjected to high temperatures, high pressures, high-speed rotation, or those that are difficult to access. Second, sensor wiring is cumbersome, leading to high maintenance costs. Finally, traditional sensors typically only measure a single point, making it difficult to capture overall vibration characteristics. These factors severely limit the application of traditional contact detection methods in complex industrial environments.

[0004] In recent years, non-contact vibration detection technology based on computer vision has developed rapidly, providing new solutions to the aforementioned problems. These methods acquire video of equipment operation using industrial cameras and extract vibration information using image processing algorithms, offering advantages such as low cost, ease of operation, and multi-point measurement capability. Among them, vibration measurement methods based on phase motion amplification have attracted widespread attention due to their high accuracy and broad applicability.

[0005] In existing technologies, such as the "A Visual Measurement Method and System for Micro-vibrations" disclosed in CN112254801A, brightness change information is converted into phase difference information, which can amplify and display micro-vibrations in videos. However, the output video contains obvious artifacts, and the vibration edges are not clear enough. The multi-scale adaptive motion magnification algorithm proposed in the paper "Multi-scale adaptive factors videoacceleration magnification" reduces artifacts and blur to some extent by manually setting the cutoff wavelength to change the magnification factor at different scales. However, it requires a large number of experiments to determine the parameters for each video and fails to achieve true adaptive processing. Although the technical solution CN118332470A achieves mobile vibration detection, it still relies on contact sensors, limiting its application scenarios.

[0006] More importantly, existing technologies generally suffer from the following problems when processing videos in real industrial environments: First, factors such as camera shake and changes in ambient light cause video instability, affecting the accuracy of vibration information extraction; second, they lack the ability to effectively decompose multi-scale and multi-directional vibrations, making it difficult to separate the main vibration from background interference; third, nonlinear distortion in the phase extraction and vibration amplification process leads to a decrease in the quality of the reconstructed video; and fourth, they lack adaptive vibration amplification strategies, making it impossible to perform targeted processing based on different frequency characteristics.

[0007] To address the aforementioned issues, particularly the critical challenge of poor stability in motion video reconstruction, there is an urgent need for a method that can effectively extract and amplify vibration information from videos in complex environments, thereby improving video reconstruction quality and vibration analysis accuracy. Summary of the Invention

[0008] To address the poor stability of motion video reconstruction, this application provides a mobile vibration inspection system and method mounted on a robotic dog. The system achieves video stabilization through the ORB algorithm and affine transformation, performs multi-scale decomposition using a complex controllable pyramid, and extracts minute vibrations using phase optical flow. This amplifies minute vibrations while maintaining video quality, thereby improving the overall video reconstruction quality.

[0009] One aspect of this application provides a mobile vibration inspection system mounted on a mechanical dog, comprising: a mobile platform for mounting the inspection system and moving within the inspection area; an image acquisition module for acquiring video images of the inspection area and preprocessing the video images; a pyramid decomposition module for performing complex controllable pyramid decomposition on the preprocessed video data, forming a bandpass filter by constructing a high-pass template, a low-pass template, and a direction template, and performing frequency domain filtering decomposition on the video to obtain the multi-scale and multi-directional amplitude and phase difference of each frame of the video; and a displacement extraction module for extracting vibration displacement based on the phase difference using the phase optical flow method to obtain the physical displacement D in the x and y directions. x and D y The frequency analysis module performs windowing processing on the vibration displacement to obtain the vibration frequency characteristics f. main The video reconstruction module, based on amplitude, phase difference, and vibration frequency characteristics f main The original video is magnified and reconstructed to obtain a reconstructed video with motion magnification.

[0010] Another aspect of this application provides a mobile vibration inspection method mounted on a mechanical dog, comprising: acquiring video data; preprocessing the video data to obtain preprocessed video data; performing complex controllable pyramid decomposition on the preprocessed video data to obtain multi-scale and multi-directional amplitude and phase difference of each frame of video; extracting vibration displacement using phase optical flow method based on the phase difference; performing Fourier transform frequency analysis on the vibration displacement to obtain vibration frequency characteristics; and amplifying and reconstructing the original video based on the obtained amplitude and phase difference, as well as the obtained vibration frequency characteristics, to obtain a reconstructed video after motion amplification.

[0011] Furthermore, the video data is preprocessed to obtain preprocessed video data, including: performing grayscale conversion and histogram equalization on the acquired video data to obtain enhanced video data; using the ORB algorithm to detect feature points on the enhanced video data to obtain feature point sets between video frames; performing feature descriptor matching on the feature point sets using Hamming distance to obtain initial matching point pairs; constructing an affine transformation matrix based on the initial matching point pairs, and optimizing the affine transformation matrix using iterative weighted least squares method to obtain the affine transformation matrix between adjacent frames; calculating the global motion trajectory based on the affine transformation matrix between adjacent frames; smoothing the global motion trajectory using the Kalman filter algorithm to obtain the smoothed global motion trajectory; and performing an affine transformation based on the smoothed global motion trajectory using the inverse mapping algorithm to obtain the preprocessed video data.

[0012] The expression for the affine transformation matrix is ​​as follows: in, It is the global affine transformation matrix of the i-th frame. Let be the local affine transformation matrix of the i-th frame, which includes translation, rotation, and scaling components.

[0013] Among them, the ORB algorithm (Oriented FAST and Rotated BRIEF algorithm) is an efficient feature point detection and description algorithm that combines the FAST keypoint detector and the BRIEF feature descriptor. In this application, the ORB algorithm is used to quickly extract feature points (such as corner points, edge intersections, etc.) from enhanced video frames and generate rotation-invariant binary feature descriptors, providing a foundation for subsequent inter-frame matching. Compared with traditional SIFT and SURF algorithms, the ORB algorithm has higher computational efficiency and is more suitable for real-time processing of video stream data on mobile platforms such as robotic dogs.

[0014] Global motion trajectory: refers to the cumulative motion path of the camera (image acquisition module on the robotic dog) throughout the entire video sequence. In this application, it is determined by matrix multiplication G. i =G i-1 ×A iThe local affine transformation A between adjacent frames i Accumulated calculations yield the global transformation matrix G for each frame relative to the reference frame (usually the first frame). i The global motion trajectory includes motion information such as translation, rotation, and scaling generated during the robot dog's movement.

[0015] Reverse mapping algorithm: an image resampling technique for reconstructing images based on a transformation matrix. In this application, the reverse mapping algorithm calculates the corresponding position of each pixel in the output image in the original image based on the smoothed global motion trajectory, and then determines the pixel value through an interpolation method (such as bilinear interpolation). Compared with forward mapping, reverse mapping can avoid the appearance of unassigned pixels ("holes") in the output image, ensuring the image quality after video stabilization.

[0016] Furthermore, the preprocessed video data is subjected to complex controllable pyramid decomposition to obtain the multi-scale and multi-directional amplitude and phase difference of each frame of video. This includes: converting each frame of the preprocessed video data from a Cartesian coordinate system to a polar coordinate system to obtain a polar coordinate image, which includes a polar radius image and a polar angle image; constructing high-pass and low-pass templates at each scale based on the polar coordinate images, where each scale corresponds to a preset polar radius value r; constructing a direction template based on the polar coordinate images; constructing a scale-direction bandpass filter based on the constructed high-pass and low-pass templates at each scale and the direction template; using the scale-direction bandpass filter to perform frequency domain filtering decomposition on each frame of the preprocessed video data to obtain frequency bands at each scale and in each direction; performing an inverse Fourier transform on each frequency band to convert the frequency domain information back to the spatial domain to obtain spatial domain information containing complex values; extracting the spatial domain phase difference from the spatial domain information, and obtaining the phase difference Δφ by calculating the difference between the phase of each frame and the phase of the first frame. ω,θ (x,t); Extract the spatial domain amplitude from the spatial domain information to obtain the amplitude of each video frame.

[0017] Polar radius image and polar angle image: These are two component images obtained after transforming the preprocessed video frame from the Cartesian coordinate system (x,y) to the polar coordinate system (r,θ). The polar radius image records the distance of each pixel from the origin (usually set as the image center); the polar angle image records the angle between each pixel and the reference axis (usually the horizontal axis).

[0018] High-pass and low-pass templates: Filter templates used for frequency selection in complex controllable pyramid decomposition. The high-pass template preserves high-frequency components (such as edges, details, and fast vibrations) and, by definition, has a radius less than r. l When the value is 1, in r l to r h Smooth transition between them, when the value is greater than r hThe value is 0 when taking the value. The low-pass template retains low-frequency components (such as the background and slow vibrations), and its definition is complementary to the high-pass template (the sum at any point of the two is 1). These templates are constructed according to different scales (i.e., different polar radius values r), enabling multi-scale analysis of vibration signals.

[0019] Direction template: A filter template used to distinguish vibrations in different directions, designed in the form of a cosine function. By dividing the 360-degree plane into N direction regions, the direction template can selectively enhance or suppress vibration signals in specific directions. This enables the system to separate and analyze vibration components from different directions in a complex industrial environment, improving the accuracy of vibration characteristic identification.

[0020] Spatial domain information: Refers to the data transformed back from the frequency domain through the inverse Fourier transform, which is in complex form and contains two key components: amplitude and phase. The amplitude represents the intensity of the vibration, and the phase contains information about the time and spatial position of the vibration. In this application, by calculating the phase difference between each frame and the reference frame (the first frame), key information about the tiny motion of the object can be extracted, which is the basic data for subsequent vibration displacement extraction and analysis.

[0021] Furthermore, the high-pass template expression is: when rad≥r, the high-pass filter function takes 1; when r×2 -twidth <rad<r, the high-pass filter function takes when rad≤r×2 -twidth the high-pass filter function takes 0; where r represents the polar radius, controlling the range of action of the high-pass template; twidth controls the non-action range; rad represents the polar angle; the low-pass template expression is: LOMASK = 1 - HIMASKA 2 ; where LOMASK represents the low-pass template function; HIMASKA represents the high-pass template function;

[0022] Furthermore, the direction template expression is: order = orientation - 1;

[0023]

[0024] where order represents the filter number; orientation represents dividing the entire plane into orientation directions; const is a normalization constant used to balance the energy of filters in different directions / orders; angle b represents the direction template in the b-th direction; angle represents the polar angle; b represents the number of the direction, usually taking 1, 2, 3,......n, orientation;

[0025] Furthermore, based on the phase difference, the vibration displacement is extracted using the phase optical flow method, including: filtering the phase difference to obtain the filtered phase difference B. ω,θ (x,t)';The filtered phase difference B ω,θ (x,t)' is converted to planar coordinate phase difference B ω,θ (x,y,t); phase difference B in planar coordinates ω,θ (x,y,t) is decomposed into a complex controllable pyramid to obtain complex subbands: Where, φ ω,θ (x, y, t) represents the local phase difference; i is the imaginary part operator in a complex number; based on the constant phase, the sub-band signal is differentiated in time to obtain the velocity components u and v; the velocity components u and v are discretely integrated, and the displacement sequences d in the x and y directions are calculated based on the video frame rate f. x (n) and d y (n); using the displacement sequence d in the x and y directions based on the amplitude. x (n) and d y (n) Perform spatial weighted averaging to obtain the noise-reduced displacement signal. and Based on the physical length L and pixel length N of the measuring component, the noise-reduced displacement signal is... and Perform a unit conversion to obtain the physical displacement D in the x and y directions. x and D y .

[0026] Among them, the complex controllable pyramid decomposition is an advanced multi-scale signal analysis technique that decomposes planar coordinate phase difference signals into complex subbands of different scales and directions. In this application, this decomposition technique can extract local phase difference information and represent it in complex form, preserving the amplitude and direction information of vibration, thus providing a mathematical basis for the subsequent accurate extraction of minute vibration displacements.

[0027] Time differentiation: The calculation of the derivative of the subband signal in the time dimension is based on the principle of constant phase. In this application, time differentiation obtains the velocity components u and v of the vibration by calculating the rate of change of complex subbands between adjacent frames. The principle of constant phase assumes that the local structure of the object remains unchanged in a short time, so the change in phase directly corresponds to the motion of the object.

[0028] Pixel length N: The number of pixels occupied by the measured component in the image, a key parameter for physical unit conversion. In this method, the pixel length N, together with the actual physical length L of the component, is used to establish the proportional relationship (L / N) between pixel displacement and physical displacement. This scaling factor enables the system to accurately convert the measured pixel-unit vibration displacement in the image into physical units (such as millimeters and micrometers) used in engineering applications.

[0029] Furthermore, the displacement sequences d in the x and y directions are calculated. x (n) and d y (n) is expressed by the following formula:

[0030]

[0031] Where u(i) represents the velocity in the x-direction, v(i) represents the velocity in the y-direction; Δt represents the time interval between frames. Where f represents the video frame rate;

[0032]

[0033] Where φ0(x,y,t) represents the phase difference in the x-direction. This represents the phase difference in the y-direction;

[0034] Furthermore, Fourier transform frequency analysis is performed on the vibration displacement to obtain the vibration frequency characteristics, including: using a Hamming window function to analyze the physical displacement D. x and D y Windowing is applied to obtain the displacement signal D. w For displacement signal D w Perform a Fast Fourier Transform to obtain the complex spectrum D(k); calculate the amplitude spectrum A(f) of the complex spectrum D(k); obtain the maximum point f of the amplitude spectrum A(f). peak Using the maximum point f peak and the maximum value point f peak Calculate the vibration frequency f at adjacent points. main ;

[0035] Furthermore, the vibration frequency f is calculated. main The following formula is used:

[0036]

[0037] Where, k peak The index of the maximum value point; Δk represents the correction amount; f s N represents the video frame rate; N represents the signal length.

[0038]

[0039] Where A(k+1) represents the amplitude at point k+1; A(k-1) represents the amplitude at point k-1; and A(k) represents the amplitude at point k.

[0040] Furthermore, based on the obtained amplitude and phase difference, as well as the acquired vibration frequency characteristics, the original video is magnified and reconstructed to obtain a motion-magnified reconstructed video, including: based on the vibration frequency f main Adjust the upper and lower frequency limits of the bandpass filter, and adjust the phase difference B according to the adjusted bandpass filter. ω,θ (x,t) is filtered to obtain the filtered phase difference B. ω,θ (x,t)'; based on the spatial wavelength λ n Calculate the magnification α of the complex controllable pyramid at different scales based on the vibration displacement δ(t) of the video image. n The vibration displacement δ here n pixel displacement and The resultant displacement: According to the magnification α n and the filtered phase difference B ω,θ (x,t)', calculate the amplified phase difference φ ω,θ (x,t)'; based on the amplified phase difference φ ω,θ The amplified complex subband is obtained by combining (x,t)' with the amplitude of the corresponding original image. Based on the amplified complex subbands, the reconstructed video after motion amplification is obtained through inverse Fourier transform. The phase difference φ after amplification is calculated. ω,θ (x,t)', using the following formula: φ ω,θ (x,t)'=φ ω,θ (x,t)+α n ×B ω,θ (x,t).

[0041] Compared to existing technologies, the advantages of this application are:

[0042] (1) This application uses the ORB algorithm for feature point detection and Hamming distance matching, combined with the affine transformation matrix optimized by iterative weighted least squares method and Kalman filter smoothing, effectively eliminating external interference factors such as camera shake and ambient light changes, laying a stable foundation for subsequent vibration information extraction, and ensuring that small vibration signals are not masked by background noise.

[0043] (2) The complex controllable pyramid structure can decompose video signals at different scales and directions. By combining high-pass templates, low-pass templates and directional templates to construct bandpass filters, it is possible to analyze vibration characteristics at different frequencies and directions at the same time, which greatly improves the ability to identify complex vibration modes. It is especially suitable for the analysis of multi-source vibrations in industrial environments.

[0044] (3) By using the principle of constant phase and time differentiation, combined with spatial weighted average optimization, the influence of background noise is effectively suppressed, and pixel-level minute displacement changes can be accurately captured. The vibration measurement accuracy is improved by converting the displacement into physical displacement through the scaling factor. Attached Figure Description

[0045] Figure 1 This is a hardware working concept diagram of this application;

[0046] Figure 2 This is a flowchart of a mobile vibration inspection method mounted on a mechanical dog, as described in this application;

[0047] Figure 3 This is the waveform of the motor vibration displacement amplitude detected in the embodiments of this application;

[0048] Figure 4 This is an analysis of the motor vibration frequency detected in the embodiments of this application. Detailed Implementation

[0049] The present application will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0050] Example 1

[0051] In terms of hardware architecture, such as Figure 1 As shown, a quadrupedal mechanical dog is used as a carrier, integrating a multi-axis active gimbal and a high-precision inertial navigation system. It supports dynamic path planning and target tracking, ensuring stable approach to the target in complex terrain, such as machine tools, pipelines, or rotating equipment. The sensing unit uses a high-speed camera, supporting wide dynamic range and low-light environments, capable of capturing micron-level vibration displacements. The computing unit uses an embedded GPU module to run image processing algorithms in real time, calculating vibration displacement, vibration frequency, and amplified vibration video, and transmitting the acquired images to a cloud control platform for equipment status assessment.

[0052] In terms of software solutions, a mobile vibration inspection system mounted on a mechanical dog is characterized by the following steps:

[0053] (1) Target recognition: The mechanical dog is controlled by a remote control. After the camera captures the target object, the device identifies the target component and automatically adjusts the pan-tilt unit to center it in the video.

[0054] (2) Video capture: After adjusting the camera parameters, video is captured.

[0055] (3) Camera shake interference elimination: First, the high-precision inertial navigation system on the robot dog collects its own attitude data in real time. Then, the vibration impact parameters such as ground vibration in the environment are obtained based on the real-time background video collected by the camera. The focus is on collecting the motion state of the robot dog during the shooting process. Combined with the image matching algorithm, the video frame is compensated by affine transformation to eliminate the environmental interference caused by the camera shake of the robot dog.

[0056] (4) Extract phase difference information: Each frame of the video is decomposed into subbands of multiple scales and directions through a complex guided pyramid. Where, φ ω,θ (x,y,t) represents local phase information, while for a specific subband (ω,θ), the phase difference is Δφ. ω,θ (x,t)=φ ω,θ (x,t)-φ ω,θ (x,t-Δt).

[0057] (5) Extraction of vibration displacement information: The vibration displacement is extracted using the phase optical flow method, based on the phase constancy equation. The velocity components u and v can be obtained, then discrete integration is performed to obtain the pixel displacement sequence of the corresponding point, and finally the physical displacement D in the horizontal and vertical directions of the point is obtained through camera calibration. x and D y .

[0058] (6) Extracting vibration frequency information: For the physical displacement D of a point obtained through camera calibration x and D y By performing a Fourier transform on the displacement sequence in a certain direction and extracting the dominant frequency, the vibration frequency f at that point can be obtained.

[0059] (7) Vibration morphology visualization: For the phase difference information B after time-domain filtering ω,θ (x,t) reflects the vibration information of a fixed frequency band. Multiplying it by the amplification factor α yields the new phase signal φ. ω,θ (x,t)'=φ ω,θ (x,t)+α×B ω,θ (x,t) combines the modified phase with the original amplitude to obtain a new complex sub-band. Finally, all sub-bands are reconstructed into a magnified video frame through an inverse complex guided pyramid transformation.

[0060] Example 2

[0061] Reference Figure 2 The specific implementation steps of this application are as follows:

[0062] Maneuver the robotic dog to find a suitable location to film the object under test. After stopping, preview the vibration video of the object. Ensure that the video is clear enough, properly exposed, and free from significant obstructions and noise interference. Start recording after previewing.

[0063] After recording, environmental interference is eliminated through multi-source data fusion. The specific steps are as follows:

[0064] 1) Call the API of the industrial camera used, connect the camera's real-time stream interface to the program, and perform video preprocessing. The preprocessing stage includes grayscale conversion and histogram equalization to enhance the robustness of feature detection in low-light environments.

[0065] 2) Combining the robot dog's own pose data and the background interference data collected by the camera, the ORB algorithm is used to detect feature points every two frames. The parameters are configured as follows: scale factor set to 1.2, supporting an eight-level pyramid scale space; the number of feature points is dynamically adjusted according to the image texture complexity, with a threshold range of 500-2000 points. Then, feature descriptor matching is performed using Hamming distance, combined with an improved MSAC algorithm to eliminate mismatches. Finally, an affine transformation model is constructed. Affine transformation is a combination of linear transformations (rotation, scaling, shearing) and translation transformations, represented in two-dimensional image processing as: in This is the linear transformation part, which controls rotation, scaling, and shearing. It is the translation component, which controls the displacement of the image in the x and y directions.

[0066] The rotation component is determined by the rotation angle θ, and its expression is:

[0067] The scaling component is the scaling factor s in the x-direction. x and y-axis scaling factor s y Decision, expression:

[0068] The shear component is determined by the horizontal shear factor j and the vertical shear factor k, and its expression is:

[0069] The matrix of the linear transformation part The expression is:

[0070]

[0071] The translation components c and f correspond to the translation amounts in the x and y directions, respectively. Therefore, the expression for the local affine transformation matrix of the i-th frame is: The transformation matrix is ​​optimized using an iterative weighted least squares method.

[0072] 3) Calculate the global motion trajectory based on the cumulative affine transformation matrices of adjacent frames:

[0073] Among them, is the global affine transformation matrix of the i-th frame, and is the local affine transformation matrix of the i-th frame, including translation, rotation and scaling components.

[0074] After calculating the complete local motion trajectory, the Kalman filter is used to smooth the motion trajectory.

[0075] 4) Apply the inverse mapping algorithm to perform affine transformation: After obtaining the coordinate matrix after affine transformation, bilinear interpolation is used to maintain image continuity, and the context-aware filling algorithm is used to repair the black edges in the edge area. Finally, image reconstruction can be achieved, and a video with reduced influence of motor micro-vibrations and environmental disturbances is output.

[0076] Perform complex steerable pyramid decomposition on the obtained video to obtain the amplitude information and phase information of multiple scales and directions for each frame of the video. The complex steerable pyramid is a multi-resolution processing of images in the frequency domain. Its essence is to perform frequency domain filtering on the original image through a set of frequency domain filters with different scales and directions, so as to obtain its frequency components at different scales and directions.

[0077] The core of the complex steerable pyramid is to construct a "scale - direction" band-pass filter. "Scale" is related to the number of decomposition levels of the pyramid, and "direction" refers to the direction of image filtering on the two-dimensional plane. Its detailed design process is as follows:

[0078] 1) Design polar coordinate images, including radial images and angular images. The essence is to convert the rectangular coordinate system into a polar coordinate system.

[0079] 2) Design high-pass templates and low-pass templates at each scale. Here, "scale" actually refers to a given radial value r. We draw a circle with radius r on the polar coordinates obtained in the first step. If the inside of the circle is retained and the outside is discarded, a low-pass template at this scale is formed, and vice versa, a high-pass template at this scale is formed.

[0080] Usually, r represents the radial distance, which is used to control the action range of the high-pass template; the parameter t width is used to control the non-action range of the high-pass template. The high-pass template HIMASK is: when rad ≥ r, the high-pass filtering function takes 1;

[0081] when r × 2 -twidth < rad < r, the high-pass filtering function takes

[0082] when rad ≤ r × 2 -twidth , the high-pass filtering function takes 0;

[0083] Where r represents the polar radius, controlling the effective range of the high-pass template; twidth controls the non-effective range; and rad represents the polar angle.

[0084] The low-pass template expression is:

[0085] LOMASK = 1-HIMASKA 2

[0086] Where LOMASK represents the low-pass template function; HIMASKA represents the high-pass template function;

[0087] 3) Design the orientation template. `orientation` is the orientation number, and the orientation template `angle` is given by the following formula:

[0088] order = orientation - 1

[0089]

[0090]

[0091] 4) Multiplying the bandpass template and direction template at each scale forms the required set of directional filters. Each frame of the input image is decomposed using these directional filters to obtain the frequency bands at each scale and direction. Then, an inverse Fourier transform is performed on each frequency band to return to the spatial domain. The resulting pixel values ​​become complex numbers, allowing us to extract the spatial domain phase. Subtracting the phase of each frame from the phase of the first frame yields the phase difference Δφ. ω,θ (x,t), where x represents the position and t represents the current frame number.

[0092] Extract vibration displacement information. For the phase difference Δφ obtained in step 3... ω,θ (x,t) contains phase difference information for all frequencies in the image. It is subject to low-frequency drift and high-frequency noise interference. A linear phase filter is used to remove the low-frequency drift and high-frequency noise, preserving the target frequency band B. ω,θ (x,t).

[0093] Vibration displacement information is extracted using the phase optical flow method, and the specific steps are as follows:

[0094] 1) For the phase difference signal B obtained after filtering ω,θ Given (x,t), if we change the position variable x to planar coordinates (x,y), then the phase difference signal is B. ω,θ (x,y,t), according to the process of complex controllable pyramid decomposition in step 3, the subband after complex controllable pyramid decomposition can be represented as:

[0095]

[0096] Where, φ ω,θ (x,y,t) represents local phase information.

[0097] 2) Based on the fact that the local phase is constant along the trajectory, i.e., the phase constant equation: φ ω,θ (x,y,t)=c;

[0098] For a certain constant c, we obtain the following from the time derivative: Where u and v are the velocities in the x and y directions, respectively, by separating the parameters of the above equation, we can obtain the expressions for u and v:

[0099]

[0100] 3) For the velocity components u and v, discrete integration yields the displacement sequence. Assuming the video frame rate is f (frames / second), the time interval between each frame is... For the nth frame, the displacements in the x and y directions are d. x (n) and d y The formula for calculating (n) is: To improve the signal-to-noise ratio of the displacement signal, spatial weighted averaging can be used for optimization. This involves using local amplitude as weights to perform a weighted average of the displacement of each pixel, resulting in a denoised displacement signal. The solution method is similar to the above formula.

[0101] 4) The displacement signal obtained in the previous step is a pixel displacement, which needs to be converted to a unit through camera calibration to obtain the real-world physical displacement. The physical length L of the component is measured using existing vision technology. Then, during displacement analysis, the pixel length N of the component is obtained based on image processing technology. Therefore, the expression for the scaling factor (scale) is:

[0102] The final physical displacement expressions in the x and y directions are:

[0103] Extract vibration frequency information. The time-domain rate of change of the displacement signal directly reflects the motion frequency, which can be obtained by analyzing the physical displacement D. x Or D y After performing a Fourier transform, the peak value of the spectrum is used to determine the dominant frequency. Here, we take D as an example. x For example, the specific steps are as follows:

[0104] 1) Data Preprocessing. A windowing function is applied to the displacement sequence to suppress spectral leakage. A Hamming window is used here, assuming a signal length of N. The processed displacement signal D... w for:

[0105] 2) For the preprocessed signal D w Perform a Fast Fourier Transform (FFT), where the number of sampling points N is a power of 2 equal to the closest signal length, and the sampling rate f... s Given the video frame rate, obtain its complex spectrum D(k):

[0106] 3) Calculate the amplitude spectrum of the signal. The amplitude spectrum of the complex spectrum represents the energy of each frequency component. The calculation method is as follows:

[0107]

[0108] 4) The dominant frequency position is initially determined by finding the maximum point of the amplitude spectrum: f peak =argmax f A(f);

[0109] At this point, the discrete frequency index k is obtained. peak .

[0110] 5) Use three adjacent points for parabolic fitting to improve resolution:

[0111] In k peak After correction by parabolic fitting of the point, the dominant frequency f main for:

[0112] Video reconstruction by zooming in. Video reconstruction by zooming in involves the following steps:

[0113] 1) For the spatial domain phase difference B obtained in step 3 ω,θ For (x,t), we need to perform a time-domain bandpass filter to extract the phase difference change corresponding to the motion we are interested in. Here, the upper and lower limits of the bandpass filter are dynamically adjusted, usually obtained by adding or subtracting 1 Hz from the main frequency obtained in step 5.

[0114] 2) For the filtered data, we multiply it by an amplification factor α and pass it through the spatial wavelength λ. n The relationship between the vibration displacement δ(n) of the video image and the adaptive adjustment of the magnification α at different scales of the complex controllable pyramid. n It satisfies the formula:

[0115]

[0116] After calculation, the amplified phase difference signal φ is obtained. ω,θ (x,t)', its expression is: φ ω,θ (x,t)'=φ ω,θ (x,t)+α n ×B ω,θ(x,t);

[0117] Next, multiplying the amplified phase difference signal by the corresponding amplitude value of the original image yields the amplified image data, i.e., the new complex subband.

[0118] 3) What we now have is the magnified image data S for each frame, representing different spatial frequency bands. ω,θ (x,t)'. According to the formula, we need to perform a Fourier transform to convert it to the frequency domain, then multiply it with the original complex controllable filter bank, and sum the results to reconstruct the complete image. Perform this operation for each frame, and finally perform an inverse Fourier transform to return to the spatial domain, thus obtaining the motion-magnified video.

[0119] The foregoing illustrative description of the present application and its embodiments is not restrictive and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. The accompanying drawings are only one embodiment of the present application, and the actual structure is not limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present application, such designs should fall within the scope of protection of this application. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" preceding an element does not exclude the inclusion of "a plurality" of that element. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A mobile vibration inspection system mounted on a mechanical dog, characterized in that, include: Mobile platform, used to install the inspection system and move within the inspection area; The image acquisition module acquires video images of the inspection area and preprocesses the video images. The pyramid decomposition module performs complex controllable pyramid decomposition on the preprocessed video data. By constructing high-pass templates, low-pass templates, and directional templates, a bandpass filter is formed to perform frequency domain filtering decomposition on the video, obtaining the multi-scale and multi-directional amplitude and phase difference of each frame of the video. The displacement extraction module uses the phase optical flow method based on the phase difference to extract the vibration displacement, obtaining the physical displacement D in the x and y directions. x and D y ; The frequency analysis module performs windowing processing on the vibration displacement to obtain the vibration frequency characteristics f. main ; The video reconstruction module, based on amplitude, phase difference, and vibration frequency characteristics f main The original video is magnified and reconstructed to obtain a reconstructed video with motion magnification.

2. A mobile vibration inspection method mounted on a mechanical dog, characterized in that, include: Collect video data; The video data is preprocessed to obtain preprocessed video data; The preprocessed video data is decomposed into a complex controllable pyramid to obtain the multi-scale and multi-directional amplitude and phase difference of each frame of video. Based on the phase difference, the vibration displacement is extracted using the phase optical flow method; Fourier transform frequency analysis is performed on the vibration displacement to obtain the vibration frequency characteristics; Based on the obtained amplitude and phase difference, as well as the acquired vibration frequency characteristics, the original video is magnified and reconstructed to obtain the reconstructed video after motion magnification.

3. The mobile vibration inspection method mounted on a mechanical dog according to claim 2, characterized in that: The preprocessed video data obtained includes: The acquired video data is subjected to grayscale conversion and histogram equalization to obtain enhanced video data. The ORB algorithm is used to detect feature points in the enhanced video data to obtain the feature point set between video frames. Initial matching point pairs are obtained by performing feature descriptor matching on the feature point set using Hamming distance; The affine transformation matrix is ​​constructed based on the initial matching point pairs, and the affine transformation matrix is ​​optimized by the iterative weighted least squares method to obtain the affine transformation matrix between adjacent frames. Calculate the global motion trajectory based on the affine transformation matrix between adjacent frames; The global motion trajectory is smoothed using the Kalman filter algorithm to obtain the smoothed global motion trajectory; Based on the smoothed global motion trajectory, an affine transformation is performed using a reverse mapping algorithm to obtain the preprocessed video data; The expression for the affine transformation matrix is ​​as follows: in, It is the global affine transformation matrix of the i-th frame. Let be the local affine transformation matrix of the i-th frame, which includes translation, rotation, and scaling components.

4. The mobile vibration inspection method mounted on a mechanical dog according to claim 3, characterized in that: The multi-scale and multi-directional amplitude and phase difference of each frame of video are obtained, including: Each frame of the preprocessed video data is converted from a Cartesian coordinate system to a polar coordinate system to obtain a polar coordinate image, which includes a polar radius image and a polar angle image. Based on the polar coordinate image, construct high-pass and low-pass templates at various scales, where each scale corresponds to a preset polar radius value r. Construct an orientation template based on the polar coordinate image; Based on the constructed high-pass and low-pass templates at each scale, as well as the directional template, a scale-directional bandpass filter is constructed. The frequency domain filtering decomposition of each frame of the preprocessed video data is performed using a scale-direction bandpass filter to obtain the frequency bands at each scale and in each direction. Perform an inverse Fourier transform on each frequency band to convert the frequency domain information back to the spatial domain, and obtain spatial domain information containing complex values. The spatial domain phase difference is extracted from the spatial domain information, and the phase difference Δφ is obtained by calculating the difference between the phase of each frame and the phase of the first frame. ω,θ (x,t); Spatial domain amplitude is extracted from spatial domain information to obtain the amplitude of each video frame.

5. The mobile vibration inspection method mounted on a mechanical dog according to claim 4, characterized in that: Qualcomm template expressions are: When rad ≥ r, the high-pass filter function takes the value 1; When r × 2 -twidth When <rad < r, the high-pass filter function takes When rad ≤ r × 2 -twidth When the high-pass filter function is set to 0; Where r represents the polar radius, controlling the effective range of the high-pass template; twidth controls the non-effective range; and rad represents the polar angle. The low-pass template expression is: LOMASK=1-HIMASK 2 Where LOMASKA represents the low-pass template function; HIMASKA represents the high-pass template function.

6. The mobile vibration inspection method mounted on a mechanical dog according to claim 4, characterized in that: The direction template expression is: order = orientation - 1 Where, order represents the filter index; orientation indicates that the entire plane is divided into orientation directions; const is a normalization constant used to balance the energy of filters of different orientations or orders; angle b This represents the direction template in the b-th direction; angle represents the polar angle.

7. The mobile vibration inspection method mounted on a mechanical dog according to claim 2, characterized in that: Vibration displacement is extracted using the phase optical flow method, including: The phase difference is filtered to obtain the filtered phase difference B. ω,θ (x,t)'; The filtered phase difference B ω,θ (x,t)' is converted to planar coordinate phase difference B ω,θ (x,y,t); Phase difference B of planar coordinates ω,θ (x,y,t) is decomposed into a complex controllable pyramid to obtain complex subbands: Where, φ ω,θ (x,y,t) represents the local phase difference; i is the imaginary part operator in complex numbers; Based on the constant phase, the sub-band signal is differentiated in time to obtain the velocity components u and v; Discretely integrate the velocity components u and v, and calculate the displacement sequences d in the x and y directions based on the video frame rate f. x (n) and d y (n); Displacement sequences d in the x and y directions using amplitude x (n) and d y (n) Perform spatial weighted averaging to obtain the noise-reduced displacement signal. and Based on the physical length L and pixel length N of the measuring component, the noise-reduced displacement signal is... and Perform a unit conversion to obtain the physical displacement D in the x and y directions. x and D y .

8. The mobile vibration inspection method mounted on a mechanical dog according to claim 6, characterized in that: Calculate the displacement sequence d in the x and y directions. x (n) and d y (n) is expressed by the following formula: Where u(i) represents the velocity in the x-direction, v(i) represents the velocity in the y-direction; Δt represents the time interval between frames. Where f represents the video frame rate; Where φ0(x,y,t) represents the phase difference in the x-direction. This represents the phase difference in the y-direction.

9. The mobile vibration inspection method mounted on a mechanical dog according to claim 7, characterized in that: Obtain vibration frequency characteristics, including: Using the Hamming window function to measure the physical displacement D x and D y Windowing is applied to obtain the displacement signal D. w ; For displacement signal D w Perform a fast Fourier transform to obtain the complex spectrum D(k); Calculate the amplitude spectrum A(f) of the complex spectrum D(k); Obtain the maximum value point f of the amplitude spectrum A(f) peak ; Using the maximum point f peak and the maximum value point f peak Calculate the vibration frequency f at adjacent points. main ; Calculate the vibration frequency f main The following formula is used: Where, k peak The index of the maximum value point; Δk represents the correction amount; f s N represents the video frame rate; N represents the signal length. Where A(k+1) represents the amplitude at point k+1; A(k-1) represents the amplitude at point k-1; and A(k) represents the amplitude at point k.

10. The mobile vibration inspection method mounted on a mechanical dog according to claim 2, characterized in that: The original video is magnified and reconstructed to obtain a motion-magnified reconstructed video, including: According to the vibration frequency f main Adjust the upper and lower frequency limits of the bandpass filter, and adjust the phase difference B according to the adjusted bandpass filter. ω,θ (x,t) is filtered to obtain the filtered phase difference B. ω,θ (x,t)'; According to the spatial wavelength λ n Calculate the magnification α of the complex controllable pyramid at different scales based on the vibration displacement δ(t) of the video image. n Vibration displacement δ n pixel displacement and The resultant displacement: According to the magnification α n and the filtered phase difference B ω,θ (x,t)', calculate the amplified phase difference φ ω,θ (x,t)'; Based on the magnified phase difference φ ω,θ The amplified complex subband is obtained by combining (x,t)' with the amplitude of the corresponding original image. Based on the magnified complex subbands, the reconstructed video after motion magnification is obtained through inverse Fourier transform; Calculate the amplified phase difference φ ω,θ (x,t)' is defined by the following formula: f ω,θ (x,t)'=φ ω,θ (x,t)+α n ×B ω,θ (x,t).

Images