A moire-based mechanical vibration detection system and method

Abstract

The application discloses a moire-based mechanical vibration detection system and method, which is applied to a computing device connected with a camera in a camera-based non-contact vibration detection scene and comprises the following modules: an image preprocessing module, which is used for selecting a region of interest from an image captured by the camera, performing image enhancement and converting the image into a binary image; a vibration curve calculation module, which is used for solving a three-axis vibration curve of a target object to be detected in the image in an object coordinate system within a time interval; and a vibration parameter estimation module, which is used for estimating a vibration amplitude and a vibration frequency of the target object to be detected in the image. In the application, moire is formed by the projection and superposition of a color filter array (CFA) in the camera and moire visual markers on an imaging plane, and the moire has extremely high sensitivity to the relative pose change between the camera and the target object.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of vibration detection, and particularly relates to a mechanical vibration detection system and method based on moire. BACKGROUND

[0002] Micro-vibration is common in daily life and industrial environments, and its detection and evaluation are of great importance. For industrial machinery, sudden changes in vibration direction, amplitude, or frequency can indicate different faults inside the equipment; long-term micro-vibration of building structures can mean potential structural damage, and even affect the health of residents. Traditional vibration sensors, such as piezoelectric accelerometers or eddy current displacement sensors, are usually directly attached to the target object, which can introduce potential resonance or change the inherent vibration state of the target object. In addition, researchers have also explored solutions based on laser, RFID, millimeter wave, etc.; however, laser equipment usually only allows single-point detection, and requires extremely high cost to ensure detection distance and accuracy at the same time, while radio frequency signals have to overcome complex multipath interference and have limited ability to detect micro-vibration amplitude. Therefore, there is an urgent need for a lightweight, high-precision, low-cost, and robust vibration detection solution to achieve micron-level vibration detection.

[0003] Existing mechanical vibration detection methods mainly include methods based on specific sensors, methods based on radio frequency signals, and methods based on traditional computer vision;

[0004] Methods based on specific sensors include contact and non-contact methods, contact detection methods include acceleration sensors, displacement sensors, piezoelectric sensors, etc., and are usually directly installed on the target object for detection. However, adding additional components to the target object can change its vibration characteristics, which may affect the normal operation of the equipment. Non-contact detection methods mainly refer to laser-based methods, high-end equipment can achieve high-precision detection of long-distance micro-vibration, but the price is high, which hinders its large-scale application in practical scenarios.

[0005] Vibration detection based on radio frequency signals mainly includes RFID-based methods and millimeter wave-based methods; RFID-based methods detect vibration by sensing the subtle changes in the backscattered radio frequency signal modulated by the vibration signal through the tag, but RFID-based solutions exhibit limited amplitude sensing capability, making it challenging to sense micro-vibration with an amplitude below 0.1mm in most mechanical vibration scenarios. Compared with RFID signals, millimeter waves have an advantage of millimeter-level wavelength, making millimeter wave-based methods more sensitive to micro-vibration. However, millimeter wave-based solutions are severely affected by multipath effects in practical industrial environments.

[0006] Visual-based vibration detection methods mainly include marker-based methods and markerless methods; marker-based methods require custom markers to be attached to the target object; these methods can achieve sub-pixel level perception through certain vibration amplification techniques, but still cannot achieve micron-level mechanical vibration detection; markerless methods use the inherent features of objects, such as corners, edges, and textures, to perceive the vibration of the object. However, these methods rely too much on the surface features of the target itself, resulting in significant differences in detection performance in different scenarios. In addition, an important limitation of traditional computer vision-based methods is that they cannot break through the frame rate limit of the camera to achieve high-frequency detection. In recent years, some researchers have adopted custom equipment consisting of a double-shutter camera, a beam splitter, an objective lens, and a cylindrical lens to achieve high-frequency vibration detection. However, the complexity and high computational requirements of custom equipment greatly hinder their usability and practicality. SUMMARY

[0007] In view of the deficiencies of the prior art described above, the purpose of the present application is to provide a mechanical vibration detection system and method based on moire patterns to solve the problems of insufficient precision, high cost, and poor robustness of existing mechanical vibration detection methods.

[0008] To achieve the above-mentioned purpose, the technical solutions adopted by the present application are as follows:

[0009] A mechanical vibration detection system based on moire patterns according to the present application is applied to a computing device connected to a camera in a non-contact vibration detection scenario based on the camera, and includes an image preprocessing module, a vibration curve calculation module, and a vibration parameter estimation module.

[0010] The image preprocessing module is used to select the region of interest from the image captured by the camera, perform image enhancement, and convert it into a binary image.

[0011] The vibration curve calculation module is used to solve the three-axis vibration curve of the target object to be detected in the image in the object coordinate system OXYZ within a time interval.

[0012] The vibration parameter estimation module is used to estimate the vibration amplitude and vibration frequency of the target object to be detected in the image.

[0013] Further, the image preprocessing module coarsely selects the region of interest according to the ArUco marker at the four corner points of the moire visual marker, divides the region of interest into four sub-regions with certain overlap, and fine-tunes each sub-region according to the moire feature robustness criterion to improve the robustness of the entire system. For each region of interest in each frame of image, image enhancement and binarization are performed.

[0014] Further, the coarse selection and division process of the region of interest in the image preprocessing module is as follows:

[0015] The image captured by the camera is roughly selected as a region of interest according to the ArUco marker, with a side length of L;

[0016] The edge of the roughly selected region of interest with a side length of L is cropped to obtain a central region of interest with a side length of 0.8L, so as to eliminate the influence of edge unstable features on the system accuracy;

[0017] The central region of interest with a side length of 0.8L is divided to obtain four sub-regions of interest with a side length of 0.5L and mutual overlap.

[0018] Further, the sub-region of interest fine-tuning process in the image preprocessing module is specifically:

[0019] For the current sub-region of interest, the first 10 frames of the video captured by the camera are selected, and the average values of all horizontal normalized frequency components u and vertical normalized frequency components v are calculated according to the spatial frequency vector f m calculated by the vibration curve calculation module, denoted as and

[0020] If the robustness degree of the moire feature is defined as: wherein, represents the closest integer to the distance The sub-region of interest side length is traversed in the range of [max(1, 0.5L-20), 0.5L], and the length that makes the robustness degree of the moire feature maximum is selected as the final sub-region of interest side length, denoted as L o ; the width and height of the final fine-tuned sub-region of interest are L i =L o and L v =L o , respectively.

[0021] If the robustness degree of the moire feature is defined as: wherein, represents the closest integer to the distance The sub-region of interest side length is traversed in the range of [max(1, 0.5L-20), 0.5L], and the length that makes the robustness degree of the moire feature maximum is selected as the final sub-region of interest side length, denoted as L o ; the width and height of the final fine-tuned sub-region of interest are L u =L o and L v =L o , respectively.

[0022] If then according to the degree of moire feature robustness fine-tune the width of the sub-region of interest in the range of [max(1, 0.5L-20), 0.5L], according to the degree of moire feature robustness fine-tune the height of the region of interest in the range of [max(1, 0.5L-20), 0.5L], and the width and height of the region of interest after final fine-tuning are L u and L v .

[0023] Further, the image enhancement and binarization process in the image preprocessing module is specifically:

[0024] performing histogram equalization on the extracted region of interest to enhance contrast;

[0025] performing Gaussian filtering on the equalized image to smooth random noise;

[0026] performing adaptive binarization on the Gaussian filtered image.

[0027] Further, the vibration curve calculation module extracts moire features for each sub-region of interest of each frame of image through a frequency spectrum weighted feature extraction algorithm: a spatial frequency vector f m (including: spatial frequency size f m and spatial frequency direction θ m ) and phase Then, the in-plane vibration model based on moire phase is used to calculate the X-axis and Y-axis vibration curves, and the off-plane vibration model based on moire spatial frequency is used to calculate the Z-axis vibration curve; a multi-channel fusion algorithm is used to fuse the data of multiple sub-regions of interest to obtain the final X-axis vibration curve x', Y-axis vibration curve y', and Z-axis vibration curve z'.

[0028] Further, the origin O of the object coordinate system OXYZ is the geometric center of the moire visual marker in the initial frame; the X-axis and Y-axis of the object coordinate system OXYZ are parallel to the long side and short side of the moire visual marker, respectively, and the Z-axis is orthogonal to the XOY plane.

[0029] Further, the moire visual marker refers to two groups of one-dimensional stripes printed on paper, which are perpendicular to each other and have the same spatial frequency; the directions of the two groups of one-dimensional stripes are by default parallel to the X-axis and Y-axis of the object coordinate system OXYZ, respectively.

[0030] Further, the process of extracting moire features in the vibration curve calculation module is specifically:

[0031] performing fast Fourier transform on the image after image preprocessing to obtain a frequency spectrum containing moire features;

[0032] The point with the maximum amplitude value in the [-45°, 45°) region in the frequency spectrum is found, and the coordinates are denoted as (u v ,v v ). The point is respectively taken as the upper left corner, the upper right corner, the lower right corner, and the lower left corner of a 2x2 region, the amplitude sums of the four 2x2 regions are respectively calculated, one 2x2 region with the maximum amplitude sum is selected as the final weighted region, and the weighted coordinates (u v_w ,v v_w ) of the brightest pulse point are calculated with the amplitude value as the weight.

[0033] The point with the maximum amplitude value in the [45°, 135°) region in the frequency spectrum is found, and the coordinates are denoted as (u h ,v h ). The point is respectively taken as the upper left corner, the upper right corner, the lower right corner, and the lower left corner of a 2x2 region, the amplitude sums of the four 2x2 regions are respectively calculated, one 2x2 region with the maximum amplitude sum is selected as the final weighted region, and the weighted coordinates (u h_w ,v h_w ) of the brightest pulse point are calculated with the amplitude value as the weight.

[0034] According to the weighted coordinates (u v_w ,v v_w ) and (u h_w ,v h_w ) of the two brightest pulse points extracted above, the moire feature calculation method in the camera coordinate system is as follows:

[0035]

[0036] Where f h and θ h represent the spatial frequency size and direction of the horizontal moire; f v and θ v represent the spatial frequency size and direction of the vertical moire; σ u and σ v are conversion coefficients for converting the normalized frequencies u and v in the frequency spectrum into absolute frequencies with real physical meaning; L c represents the pixel unit size of the CFA; L u and L v represent the width and height of the region of interest in units of pixels.

[0037] The mode of (u h ,v h ) and (u v ,v v ) calculated from the first 10 frames of images is denoted as (u h_m ,v h_m ) and (uv_m ,v v_m ), extract the phase at the (u h_m ,v h_m ) and (u v_m ,v v_m ) coordinates of each frame spectrum as the phase of the entire image, denoted as and

[0038] and represent the two groups of moire features calculated, denoted as horizontal moire features and vertical moire features, respectively.

[0039] Further, the in-plane vibration model based on moire phase in the vibration curve calculation module is:

[0040] Calculate the displacement Δx and Δy of the object in the X-axis and Y-axis between adjacent frames in turn, and the calculation method is as follows:

[0041]

[0042] where T code represents the spatial period size of the moire visual mark, represents the phase in the difference between adjacent frames, represents the phase in the difference between adjacent frames;

[0043] Set the X-axis coordinate and Y-axis coordinate of the object in the initial frame as 0, according to the X-axis coordinate of the next frame is equal to the X-axis coordinate of the previous frame plus the X-axis displacement Δx, and the Y-axis coordinate of the next frame is equal to the Y-axis coordinate of the previous frame plus the Y-axis displacement Δy, the X-axis vibration curve x and the Y-axis vibration curve y of the object are calculated respectively.

[0044] Further, the off-plane vibration model based on moire spatial frequency in the vibration curve calculation module is:

[0045] The distance from the camera imaging plane to the XOY plane of the object coordinate system is calculated as follows:

[0046]

[0047] where d h is the distance from the camera to the XOY plane of the object coordinate system calculated according to the horizontal moire features; d v is the distance from the camera to the XOY plane of the object coordinate system calculated according to the vertical moire features; f is the focal length of the camera, f code is the spatial frequency of the moire visual mark, f cθ represents the spatial frequency of the camera color filter array CFA. c The spatial frequency direction of the CFA in the object coordinate system;

[0048] The calculated distance d h and d v The mean value d is calculated to enhance the object's inherent Z-axis vibration mode while eliminating random noise. The calculation method is as follows:

[0049]

[0050] The distance d corresponding to each frame of the image is obtained by iterating through the images. The distance d0 corresponding to the initial frame is then subtracted to obtain the Z-axis vibration curve z in the object coordinate system OXYZ.

[0051] The above calculation results x, y, and z are the X-axis, Y-axis, and Z-axis vibration curves obtained from one sub-region of interest. The above steps are repeated for each of the four sub-regions of interest, and the triaxial vibration curves obtained from the i-th region of interest are denoted as x, y, and z, respectively. i y i and z i , where i = 1, 2, 3, 4.

[0052] Furthermore, the multi-channel fusion algorithm of the vibration curve calculation module is specifically as follows:

[0053] The arithmetic mean of the data obtained from the four sub-regions of interest is calculated to obtain the final triaxial vibration curves x′, y′, and z′ of the object, which are as follows:

[0054]

[0055] Furthermore, the method for calculating the vibration amplitude in the vibration parameter estimation module is as follows:

[0056] The X-axis vibration amplitude A is calculated based on the X-axis vibration curve x′. x The calculation method is as follows:

[0057] x normal =x′-mean(x′)

[0058]

[0059] Where mean(x′) represents the result of calculating the mean of the time series data x′; x normal The normalized data for x′; For time series data x normal The i-th data in; rms x This represents the effective value of the amplitude.

[0060] The Y-axis vibration amplitude A is determined based on the Y-axis vibration curve y′.y The calculation method is as follows:

[0061] y normal = y'-mean(y')

[0062]

[0063] wherein mean(y') represents the result of averaging the time series data y'; y normal is the normalized data of y'; is the i-th data in the time series data y normal ; rms y is the root mean square value;

[0064] The Z-axis vibration amplitude A is solved based on the Z-axis vibration curve z' z , and the calculation method is as follows:

[0065] z normal = z'-mean(z')

[0066]

[0067] wherein mean(z') represents the result of averaging the time series data z'; z normal is the normalized data of z'; is the i-th data in the time series data z normal ; rms z is the root mean square value.

[0068] Further, the calculation method of the vibration frequency in the vibration parameter estimation module is:

[0069] The largest prime number in the adjustable frame rate range of the camera is selected as the first frame rate fps1, and the corresponding three-axis vibration curves x fps1 , y fps1 and z fps1 are obtained, the fast Fourier transform is performed on the three-axis vibration curves x fps1 , y fps1 and z fps1 , respectively, to obtain amplitude spectra X fps1 , Y fps1 and Z fps1 , and the frequencies corresponding to the points with the largest amplitude in the amplitude spectra X fps1 , Y fps1 and Z fps1 are recorded as observation frequencies obs x1 , obs y1 and obs z1 , respectively; three observation frequencies obs x1 , obs y1 and obs z1The mean of obs1 is denoted as obs1;

[0070] All possible real frequencies under the observation frequency obs1 are denoted as Where frequency max is set to the maximum real frequency that may occur in the application scenario, b i The calculation formula of obs1 is as follows:

[0071]

[0072] Where b1=obs1; b2=fps1-obs1;

[0073] The maximum deviation between the ideal value and the actual value of the real frequency is set as ε, and the interval of all possible real frequencies under the observation frequency obs1 is

[0074] The second frame rate fps2 is selected according to the double frame rate selection algorithm, and the double frame rate selection algorithm is as follows:

[0075] The second frame rate fps2 is selected from the set For each frame rate in the set A, the possible real frequency interval is calculated, and the interval of all observation frequencies under this frame rate is selected. The frame rate with the maximum interval of is selected as fps2.

[0076] Based on the second frame rate fps2, the corresponding three-axis vibration curves x fps2 , y fps2 and z fps2 are obtained, and fast Fourier transform is performed on the three-axis vibration curves x fps2 , y fps2 and z fps2 to obtain amplitude spectra X fps2 , Y fps2 and Z fps2 . fps2 The frequency corresponding to the point with the maximum amplitude in the amplitude spectrum X fps2 , Y fps2 and Z x2 is respectively denoted as the observation frequency obs y2 , obs z2 , and obs x2 . The mean of the three observation frequencies obs y2 , obs z2 is denoted as obs2.

[0077] The vibration frequency is calculated based on the double frame rate frequency reconstruction method, and the double frame rate frequency reconstruction method is as follows:

[0078] According to the relationship among the real frequency, the observed frequency and the camera frame rate, all possible real frequencies corresponding to the observed frequencies obs1 and obs2 in the maximum frequency range [0, frequency max ] are found, and are denoted as and respectively. The relationship among the real frequency F t , the observed frequency F o and the camera frame rate fps is as follows:

[0079]

[0080] wherein MOD(·) is a remainder function;

[0081] When there are i and j such that a i and c j satisfy the relationship |a i -c j |<ε, it is considered that a i and c j are equal, and ε is the maximum deviation between the ideal value and the actual value of the real frequency; the calculation formula of the vibration frequency F is as follows:

[0082]

[0083] The mechanical vibration detection in the application refers to detecting the vibration state of a target object with a pasted Moire visual mark by using a camera; it is suitable for the scene where the camera is fixed and the target object moves. The vibration state of the target object is described as the three-axis vibration curves of the target object in the object coordinate system OXYZ: the X-axis vibration curve x', the Y-axis vibration curve y' and the Z-axis vibration curve z', and the vibration parameters: the three-axis vibration amplitudes A x , A y , A z and the vibration frequency F; the application can output the three-axis vibration curves of the target object in the object coordinate system, the three-axis vibration amplitudes and the vibration frequency.

[0084] The application also provides a mechanical vibration detection method based on Moire, based on the above system, including the following steps:

[0085] 1) using a camera to collect video of a target object with a pasted Moire visual mark at a first frame rate fps1, the video frame containing a Moire image with a low spatial frequency;

[0086] 2) selecting the region of interest in the video frame sequence according to the ArUco mark, dividing it into 4 sub-regions of interest, and adjusting the size of the region of interest according to the robustness degree of the Moire feature;

[0087] 3) performing image enhancement and binarization operation on the video frame sequence;

[0088] 4) For each sub region of interest, the moire features are extracted from each frame of the video by a lightweight spectral weighted feature extraction algorithm, and the in-plane vibration curves of X-axis and Y-axis are calculated by a moire phase based in-plane vibration model; the out-of-plane vibration curve of Z-axis is calculated by a moire spatial frequency based out-of-plane vibration model;

[0089] 5) The data of the four sub regions of interest are fused based on a multi-channel fusion algorithm to obtain the fused three-axis vibration curves;

[0090] 6) The three-axis vibration amplitudes are estimated according to the three-axis vibration curves obtained in step 5);

[0091] 7) The second frame rate fps2 is selected based on a double frame rate selection algorithm to perform video acquisition, and steps 2) to 5) are repeated;

[0092] 8) The vibration frequency is estimated based on a double frame rate frequency reconstruction method.

[0093] Advantages of the present application:

[0094] 1. Ultra-high precision: the moire formed by the projection superposition of the color filter array CFA in the camera and the moire visual marker on the imaging plane has extremely high sensitivity to the relative pose change between the camera and the target object; the movement of the target object in the XOY plane of the object coordinate system will cause significant changes in the moire phase characteristics; the movement of the target object along the Z-axis direction in the object coordinate system will cause significant changes in the moire spatial frequency characteristics; the spectral weighted feature extraction algorithm in the vibration curve calculation module extracts high-precision moire features, and then through the in-plane vibration model based on the moire phase, the out-of-plane vibration model based on the moire spatial frequency and the multi-channel fusion algorithm, the amplitude sensing precision of the system can reach microns, and the minimum vibration amplitude that can be sensed is 2um.

[0095] 2. Robustness: compared with the traditional feature-based computer vision method, the statistical characteristics of the moire spline in the frequency domain can provide more accurate and more robust information for vibration detection, and the sub region of interest fine tuning in the image preprocessing module of the present application can further improve the robustness of the system; the number of feature points of the traditional computer vision method is limited and is easily affected by the environment.

[0096] 3. Low cost: the present application only needs an entry-level industrial camera and a moire visual marker printed on ordinary paper to realize high-precision mechanical vibration detection, and the cost is low.

[0097] 4. High-frequency vibration detection capability: Based on the double-frame-rate frequency reconstruction and double-frame-rate selection method of the vibration parameter restoration module of the application, the limitation of the Nyquist sampling theorem can be broken through, and high-frequency vibration up to 10 times the frame rate of the camera can be detected. BRIEF DESCRIPTION OF DRAWINGS

[0098] Figure 1 The architecture diagram of the mechanical vibration detection system in the embodiment of the application;

[0099] Figure 2 The schematic diagram of the moire visual mark in the application;

[0100] Figure 3 The schematic diagram of the image region of interest rough selection and division in the application;

[0101] Figure 4 The flowchart of the image region of interest fine-tuning in the application;

[0102] Figure 5 The principle diagram of the moire feature extraction in the application;

[0103] Figure 6 The flowchart of the in-plane vibration detection method based on the moire phase in the application;

[0104] Figure 7 The flowchart of the out-of-plane vibration detection method based on the moire spatial frequency in the application;

[0105] Figure 8 The principle diagram of the multi-channel fusion algorithm in the application;

[0106] Figure 9 The flowchart of the vibration amplitude calculation method in the application;

[0107] Figure 10 The flowchart of the vibration frequency calculation method in the application. DETAILED DESCRIPTION

[0108] In order to facilitate the understanding of those skilled in the art, the application will be further described below in conjunction with the embodiments and the drawings. The content mentioned in the embodiments is not a limitation of the application.

[0109] Referring to Figures 1 to 2 The mechanical vibration detection system based on moire of the application is applied to a computing device connected with a camera in a camera-based non-contact vibration detection scene, and includes an image preprocessing module, a vibration curve calculation module, and a vibration parameter estimation module.

[0110] The image preprocessing module is used to select a region of interest (ROI) from the image captured by the camera, perform image enhancement, and convert it into a binary image.

[0111] Specifically, the image preprocessing module coarsely selects the region of interest according to the ArUco mark (positioning mark) of the Moire visual mark at the four corner points, divides the region of interest into four sub-regions of interest with a certain overlap, fine-tunes each sub-region of interest according to the Moire feature robustness criterion to improve the robustness of the entire system, and performs image enhancement and binarization for each region of interest of each frame of image.

[0112] Referring to Figure 3 The coarse selection and division process of the region of interest in the image preprocessing module is specifically as follows:

[0113] The region of interest is coarsely selected according to the ArUco mark in the image captured by the camera, and the side length is L (unit: pixel);

[0114] The edge of the coarsely selected region of interest with a side length of L is cropped to obtain a central region of interest with a side length of 0.8L, so as to eliminate the influence of unstable edge features on the accuracy of the system;

[0115] The central region of interest with a side length of 0.8L is divided to obtain four sub-regions of interest with a side length of 0.5L, which overlap with each other.

[0116] Referring to Figure 4 The fine-tuning process of the sub-region of interest in the image preprocessing module is specifically as follows:

[0117] For the current sub-region of interest, the first 10 frames of the video captured by the camera are selected, and the average values of all horizontal normalized frequency components u and vertical normalized frequency components v are calculated according to the spatial frequency vector f m calculated by the vibration curve calculation module, denoted as and

[0118] If the Moire feature robustness degree is defined as: wherein, represents the distance closest to the integer; the sub-region of interest side length is traversed in the range of [max(1, 0.5L-20), 0.5L], and the length that makes the Moire feature robustness degree maximum is selected as the final sub-region of interest side length, denoted as L o ; the width and height of the final fine-tuned sub-region of interest are L u =L o and L v =L o , respectively.

[0119] If the Moire feature robustness degree is defined as: wherein, denotes distance the nearest integer;traverse the length of the sub-region of interest in the range of [max(1, 0.5L-20), 0.5L], select the length that makes the moire feature robustness degree maximum as the final length of the sub-region of interest, denoted as L o ; the width and height of the final fine-tuned sub-region of interest are L u = L o and L v = L o ;

[0120] If , according to the moire feature robustness degree , the width of the sub-region of interest is fine-tuned in the range of [max(1, 0.5L-20), 0.5L], according to the moire feature robustness degree , the height of the sub-region of interest is fine-tuned in the range of [max(1, 0.5L-20), 0.5L], and the width and height of the final fine-tuned sub-region of interest are L u and L v .

[0121] Specifically, the image enhancement and binarization process in the image preprocessing module is specifically:

[0122] Performing histogram equalization on the extracted region of interest to enhance contrast;

[0123] Performing Gaussian filtering on the equalized image to smooth random noise;

[0124] Performing adaptive binarization on the Gaussian filtered image.

[0125] A vibration curve calculation module is configured to solve the three-axis vibration curve of the target object to be detected in the object coordinate system OXYZ in a time interval;

[0126] Specifically, for each sub-region of interest of each frame of image, the vibration curve calculation module extracts the moire feature through a frequency spectrum weighted feature extraction algorithm: a spatial frequency vector f m (including: spatial frequency size f m and spatial frequency direction θ m ) and phase Then, the vibration curve of the X-axis and the Y-axis is calculated through a moire phase based in-plane vibration model, and the vibration curve of the Z-axis is calculated through a moire spatial frequency based out-of-plane vibration model;The data of multiple sub-regions of interest is fused to obtain the final X-axis vibration curve x', Y-axis vibration curve y' and Z-axis vibration curve z' by using a multi-channel fusion algorithm.

[0127] Specifically, the origin O in the object coordinate system OXYZ is the geometric center of the moire visual mark in the initial frame; the X axis and the Y axis of the object coordinate system OXYZ are respectively parallel to the long side and the short side of the moire visual mark, and the Z axis is orthogonal to the XOY plane.

[0128] Specifically, the moire visual mark refers to two groups of one-dimensional stripes printed on paper, which are perpendicular to each other and have the same spatial frequency; the directions of the two groups of one-dimensional stripes are respectively parallel to the X axis and the Y axis of the object coordinate system OXYZ by default.

[0129] Referring to Figure 5 The process of extracting the moire features in the vibration curve calculation module is specifically as follows:

[0130] Performing fast Fourier transform on the image after image preprocessing to obtain a frequency spectrum graph containing moire features;

[0131] Finding a point with the maximum amplitude value in the [-45°, 45°) region in the frequency spectrum graph, and marking the coordinates as (u v ,v v ), taking the point as the upper left corner, the upper right corner, the lower right corner, and the lower left corner of a 2x2 region respectively, calculating the amplitude sum of the four 2x2 regions respectively, selecting a 2x2 region with the maximum amplitude sum as the final weighted region, and calculating the weighted coordinates (u v_w ,v v_w ) of the brightest impulse point with the amplitude value as the weight;

[0132] Finding a point with the maximum amplitude value in the [45°, 135°) region in the frequency spectrum graph, and marking the coordinates as (u h ,v h ), taking the point as the upper left corner, the upper right corner, the lower right corner, and the lower left corner of a 2x2 region respectively, calculating the amplitude sum of the four 2x2 regions respectively, selecting a 2x2 region with the maximum amplitude sum as the final weighted region, and calculating the weighted coordinates (u h_w ,v h_w ) of the brightest impulse point with the amplitude value as the weight;

[0133] According to the weighted coordinates (u v_w ,v v_w ) and (u h_w ,v h_w ) of the two brightest impulse points extracted above, the moire feature calculation method in the camera coordinate system is as follows:

[0134]

[0135] Wherein, f h and θ h represent the spatial frequency size and direction of the horizontal moire; fv and θ v denote the spatial frequency magnitude and direction of the vertical moire; σ u and σ v are conversion coefficients to convert the normalized frequencies u and v in the spectrum map to absolute frequencies with real physical meaning, L c denotes the pixel unit size of the CFA; L u and L v denote the width and height of the region of interest in pixels, respectively;

[0136] The mode of (u h ,v h ) and (u v ,v v ) extracted from the first 10 frames of images is calculated, denoted as (u h_m ,v h_m ) and (u v_m ,v v_m ), and the phase at the (u h_m ,v h_m ) and (u v_m ,v v_m ) coordinates of each frame spectrum map is extracted as the phase of the entire image, denoted as and

[0137] and denote the two groups of moire features calculated, denoted as horizontal moire features and vertical moire features, respectively.

[0138] Referring to FIG. 6, the in-plane vibration model based on the moire phase in the vibration curve calculation module is as follows: Figure 6

[0139] The displacement Δx and Δy of the object in the X axis and Y axis between adjacent frames are calculated in turn, and the calculation method is as follows:

[0140]

[0141] wherein, T code denotes the spatial period size of the moire visual mark, denotes the difference in phase between adjacent frames, denotes the difference in phase between adjacent frames;

[0142] The X axis coordinate and Y axis coordinate of the object in the initial frame are set to 0, and the X axis coordinate of the next frame is equal to the sum of the X axis coordinate of the previous frame and the X axis displacement Δx, and the Y axis coordinate of the next frame is equal to the sum of the Y axis coordinate of the previous frame and the Y axis displacement Δy, so as to calculate the X axis vibration curve x and the Y axis vibration curve y of the object, respectively.​

[0143] Referring to Figure 7 The out-of-plane vibration model based on the moire spatial frequency in the vibration curve calculation module is as shown in the formula (1):

[0144] The distance from the camera imaging plane to the object coordinate system XOY plane is calculated according to the formula (2):

[0145]

[0146] Wherein, d h is the distance from the camera to the object coordinate system XOY plane calculated according to the horizontal moire feature; d v is the distance from the camera to the object coordinate system XOY plane calculated according to the vertical moire feature; f is the focal length of the camera, f code is the spatial frequency of the moire visual marker, f c is the spatial frequency of the camera color filter array CFA, θ c is the spatial frequency direction of the CFA in the object coordinate system;

[0147] The mean value d of the calculated distances d h and d v is calculated, which eliminates random noise and enhances the inherent Z-axis vibration mode of the object, and the calculation method is as follows:

[0148]

[0149] The distance d corresponding to each frame of image is obtained by iteration, and the distance d0 corresponding to the initial frame is subtracted to obtain the Z-axis vibration curve z in the object coordinate system OXYZ;

[0150] The above calculation results x, y and z are the X-axis, Y-axis and Z-axis vibration curves calculated for one sub-region of interest, and the above steps are repeated for the four sub-regions of interest. The three-axis vibration curves calculated from the i-th region of interest are denoted as x i , y i and z i , wherein i = 1, 2, 3, 4.

[0151] Referring to Figure 8 The multi-channel fusion algorithm of the vibration curve calculation module is as follows:

[0152] The data calculated from the four sub-regions of interest are arithmetically averaged to obtain the final three-axis vibration curves x', y' and z' of the object, which are as follows:

[0153]

[0154]

[0155] a vibration parameter estimation module for estimating the vibration amplitude and the vibration frequency of the target object to be detected in the image.

[0156] Referring to Figure 9 the calculation method of the vibration amplitude in the vibration parameter estimation module is as follows:

[0157] solving the X-axis vibration amplitude A based on the X-axis vibration curve x′ x , the calculation method is as follows:

[0158] x normal =x′-mean(x′)

[0159]

[0160] wherein mean(x′) represents the result of averaging the time series data x′; x normal is the normalized data of x′; is the i-th data in the time series data x normal ; rms x is the effective value of the amplitude;

[0161] solving the Y-axis vibration amplitude A based on the Y-axis vibration curve y′ y , the calculation method is as follows:

[0162] y normal =y′-mean(y′)

[0163]

[0164] wherein mean(y′) represents the result of averaging the time series data y′; y normal is the normalized data of y′; is the i-th data in the time series data y normal ; rms y is the effective value of the amplitude;

[0165] solving the Z-axis vibration amplitude A based on the Z-axis vibration curve z′ z , the calculation method is as follows:

[0166] z normal =z′-mean(z′)

[0167]

[0168] wherein mean(z′) represents the result of averaging the time series data z′; z normal is the normalized data of z′; is the i-th data in the time series data z normal ; rms zThe amplitude effective value.

[0169] Referring to Figure 10 The calculation method of the vibration frequency in the vibration parameter estimation module is as follows:

[0170] Select the largest prime number in the adjustable frame rate range of the camera as the first frame rate fps1, and obtain the corresponding three-axis vibration curve x fps1 , y fps1 , and z fps1 Perform fast Fourier transform on the three-axis vibration curves x fps1 , y fps1 , and z fps1 , respectively, to obtain amplitude spectra X fps1 , Y fps1 , and Z fps1 The frequency corresponding to the point with the largest amplitude value in the amplitude spectrum X fps1 , Y fps1 , and Z fps1 is recorded as the observed frequency obs x1 , obs y1 , and obs z1 The mean of the three observed frequencies obs x1 , obs y1 , and obs z1 is obs1.

[0171] All possible true frequencies under the observed frequency obs1 are recorded as where frequency max is set as the maximum true frequency that may occur in the application scenario, and b i is calculated according to the following formula:

[0172]

[0173] where b1 = obs1; and b2 = fps1-obs1.

[0174] Set the maximum deviation between the ideal value and the actual value of the true frequency as ε, and the interval of all possible true frequencies under the observed frequency obs1 is

[0175] Select the second frame rate fps2 according to the double-frame rate selection algorithm, and the double-frame rate selection algorithm is as follows:

[0176] The second frame rate fps2 is selected from the set For each frame rate in the set A, calculate the possible true frequency interval All observed frequency intervals Select the frame rate that makes the interval in the largest interval as fps2.

[0177] Based on the second frame rate fps2, the corresponding triaxial vibration curve x is obtained. fps2 y fps2 and z fps2 For the triaxial vibration curve x fps2 y fps2 and z fps2 Perform fast Fourier transforms on each to obtain the amplitude spectrum X. fps2 ,Y fps2 Z fps2 , X amplitude spectrum fps2 ,Y fps2 Z fps2 The frequencies corresponding to the points with the largest amplitude values ​​are denoted as the observation frequencies obs. x2 obs y2 obs z2 Three observation frequencies (obs) x2 obs y2 obs z2 The mean is denoted as obs2;

[0178] The vibration frequency was calculated based on the dual-frame-rate frequency reconstruction method, which is as follows:

[0179] Based on the relationship between the actual frequency, the observed frequency, and the camera frame rate, within the maximum frequency range [0, frequency]... max Find all possible true frequencies corresponding to the observed frequencies obs1 and obs2 within the range, and denote them as follows: and True frequency F t Observation frequency F o The relationship between the camera frame rate (fps) and the camera frame rate is as follows:

[0180]

[0181] Where MOD(·) is the remainder function;

[0182] When there exist i and j such that a i and c j Satisfying relation |a i -c j When |<ε, we consider a i and c j They are equal, where ε is the maximum deviation between the ideal and actual values ​​of the true frequency; the formula for calculating the vibration frequency F is as follows:

[0183]

[0184] The mechanical vibration detection in the application refers to detecting the vibration state of a target object with a camera and a visual mark of moire pasted on the target object; the application is suitable for the scene of camera fixation and target object movement. The vibration state of the target object is described as three-axis vibration curves of the target object in the object coordinate system OXYZ, i.e., an X-axis vibration curve x', a Y-axis vibration curve y' and a Z-axis vibration curve z', and vibration parameters, i.e., three-axis vibration amplitudes A x 、A y 、A z and a vibration frequency F; the application can output the three-axis vibration curves of the target object in the object coordinate system, the three-axis vibration amplitudes and the vibration frequency.

[0185] The application also provides a mechanical vibration detection method based on moire, based on the above system, comprising the following steps:

[0186] 1) using a camera to collect video of a target object with a visual mark of moire pasted on the surface of the target object at a first frame rate fps1 (the maximum prime number in the adjustable frame rate range of the camera), and the video frame contains a moire image with a low spatial frequency;

[0187] 2) selecting a region of interest in the video frame sequence according to the ArUco mark, dividing it into four sub-regions of interest, and adjusting the size of the region of interest according to the robustness of the moire feature;

[0188] 3) performing image enhancement and binarization operations on the video frame sequence;

[0189] 4) for each sub-region of interest, extracting the moire feature of each frame of the video through a lightweight spectral weighted feature extraction algorithm, calculating the vibration curves of the X-axis and the Y-axis through a vibration model in the plane based on the phase of the moire, and calculating the vibration curve of the Z-axis through a vibration model out of the plane based on the spatial frequency of the moire;

[0190] 5) fusing the data of the four sub-regions of interest based on a multi-channel fusion algorithm to obtain the fused three-axis vibration curves;

[0191] 6) estimating the three-axis vibration amplitudes according to the three-axis vibration curves obtained in step 5);

[0192] 7) selecting a second frame rate fps2 for video collection based on a double-frame rate selection algorithm, and repeating steps 2) to 5);

[0193] 8) estimating the vibration frequency based on a double-frame rate frequency reconstruction method.

[0194] The application has many specific application approaches, and the above description is only the preferred embodiment of the application, and it should be pointed out that, for ordinary skilled in the art, several improvements can be made without departing from the principles of the application, and these improvements should also be considered as the protection scope of the application.

Claims

1. A mechanical vibration detection system based on moiré patterns, applied in a camera-connected computing device in a camera-based non-contact vibration detection scenario, characterized in that, include: Image preprocessing module, vibration curve calculation module, and vibration parameter estimation module; The image preprocessing module is used to select the region of interest from the image captured by the camera, perform image enhancement, and convert it into a binary image; The vibration curve calculation module is used to solve the triaxial vibration curve of the target object to be detected in the image within a certain time interval in the object coordinate system OXYZ. The vibration parameter estimation module is used to estimate the vibration amplitude and vibration frequency of the target object to be detected in the image; The image preprocessing module coarsely selects the region of interest (ROI) based on the ArUco markers at the four corners of the moiré visual markers, divides the ROI into four sub-ROIs with some overlap, and fine-tunes each sub-ROI according to the robustness criterion of moiré features to improve the robustness of the entire system. For each ROI in each frame of the image, image enhancement and binarization are performed. The vibration curve calculation module extracts moiré features for each sub-region of interest in each frame of image using a spectral weighted feature extraction algorithm: spatial frequency vector f. m and phase The vibration curves of the X and Y axes are then calculated using an in-plane vibration model based on moiré phase, and the vibration curve of the Z axis is calculated using an out-of-plane vibration model based on moiré spatial frequency. The data from multiple sub-regions of interest are then fused using a multi-channel fusion algorithm to obtain the final X-axis vibration curve x′, Y-axis vibration curve y′, and Z-axis vibration curve z′. The vibration parameter estimation module estimates the three-axis vibration amplitude based on the three-axis vibration curves and estimates the vibration frequency based on the dual-frame-rate frequency reconstruction method.

2. The mechanical vibration detection system based on moiré patterns according to claim 1, characterized in that, The process of coarse selection and segmentation of the region of interest in the image preprocessing module is as follows: The region of interest with side length L is coarsely selected from the images captured by the camera based on ArUco markers. The edges of the coarsely selected region of interest with a side length of L are trimmed to obtain a central region of interest with a side length of 0.8L, in order to eliminate the influence of unstable edge features on the system accuracy; The central region of interest with a side length of 0.8L is divided into four overlapping sub-regions of interest with a side length of 0.5L.

3. The mechanical vibration detection system based on moiré patterns according to claim 2, characterized in that, The sub-region of interest fine-tuning process in the image preprocessing module is as follows: For the current sub-region of interest, the first 10 frames of the video captured by the camera are selected, and the spatial frequency vector f is calculated based on the vibration curve calculation module. m To calculate the average of all horizontally normalized frequency components u and vertically normalized frequency components v, denoted as […]. and like The robustness of moiré pattern features is then defined as: in, Indicates distance The closest integer; iterate through the sub-region of interest within the range [max(1,0.5L-20), 0.5L], and select the length that maximizes the robustness of the moiré pattern feature as the final sub-region of interest side length, denoted as L. o After final fine-tuning, the width and height of the sub-region of interest are L and L, respectively. u =L o and L v =L o ; like The robustness of moiré pattern features is then defined as: in, Indicates distance The closest integer; iterate through the sub-region of interest within the range [max(1,0.5L-20), 0.5L], and select the length that maximizes the robustness of the moiré pattern feature as the final sub-region of interest side length, denoted as L. o After final fine-tuning, the width and height of the sub-region of interest are L and L, respectively. u =L o and L v =L o ; like Based on the robustness of moiré pattern characteristics The width of the region of interest is fine-tuned within the range of [max(1,0.5L-20), 0.5L], depending on the robustness of the moiré pattern features. The height of the region of interest was fine-tuned within the range of [max(1,0.5L-20), 0.5L]. The final width and height of the region of interest after fine-tuning were L... u and L v .

4. The mechanical vibration detection system based on moiré patterns according to claim 1, characterized in that, The process of extracting moiré features in the vibration curve calculation module is as follows: Perform a Fast Fourier Transform on the image after image preprocessing to obtain a spectrum containing moiré pattern features; Find the point with the largest amplitude value within the region [-45°, 45°) in the spectrum, denoted as (u... v ,v v Using this point as the top-left, top-right, bottom-right, and bottom-left corners of a 2×2 region, the amplitude sums of the four 2×2 regions are calculated. The 2×2 region with the largest amplitude sum is selected as the final weighted region. Using the amplitude value as the weight, the weighted coordinates (u) of the brightest pulse point are calculated. v_w ,v v_w ); Find the point with the largest amplitude value within the region [45°, 135°) in the spectrum, denoted as (u... h ,v h Using this point as the top-left, top-right, bottom-right, and bottom-left corners of a 2×2 region, the amplitude sums of the four 2×2 regions are calculated. The 2×2 region with the largest amplitude sum is selected as the final weighted region. Using the amplitude value as the weight, the weighted coordinates (u) of the brightest pulse point are calculated. h_w ,v h_w ); Based on the weighted coordinates (u) of the two brightest pulse points extracted above v_w ,v v_w ) and (u h_w ,v h_w The moiré pattern feature calculation method in the camera coordinate system is as follows: Among them, f h and θ h Indicates the spatial frequency magnitude and direction of horizontal moiré patterns; f v and θ v Indicates the spatial frequency magnitude and direction of vertical moiré patterns; σ u and σ v To convert the normalized frequencies u and v in the spectrum into absolute frequencies with real physical meaning, L c Indicates the pixel unit size of the CFA; L u and L v These represent the width and height of the region of interest in pixels, respectively. Calculate the (u) extracted from the first 10 frames of images h ,v h ) and (u v ,v v The mode of (u) is denoted as (u h_m ,v h_m ) and (u v_m ,v v_m Extract the (u) of the spectrogram of each frame. h_m ,v h_m ) and (u v_m ,v v_m The phase at coordinate ) is taken as the phase of the entire image, denoted as . and and The two sets of moiré patterns obtained from the calculation are denoted as horizontal moiré patterns and vertical moiré patterns, respectively.

5. The mechanical vibration detection system based on moiré patterns according to claim 4, characterized in that, The in-plane vibration model based on moiré phase in the vibration curve calculation module is as follows: The displacements Δx and Δy of objects along the X and Y axes between adjacent frames are calculated sequentially, as follows: Among them, T code The size of the fringe spatial period of the visual markers of Moore. Indicates phase The difference between adjacent frames, Indicates phase The difference between adjacent frames; The X-axis and Y-axis coordinates of the object are set to 0 in the initial frame. Based on the fact that the X-axis coordinate of the next frame is equal to the sum of the X-axis coordinate and X-axis displacement Δx of the previous frame, and the Y-axis coordinate of the next frame is equal to the sum of the Y-axis coordinate and Y-axis displacement Δy of the previous frame, the X-axis vibration curve x and the Y-axis vibration curve y of the object are calculated respectively.

6. The mechanical vibration detection system based on moiré patterns according to claim 5, characterized in that, The out-of-plane vibration model based on moiré spatial frequency in the vibration curve calculation module is as follows: The formula for calculating the distance from the camera's imaging plane to the object's XOY coordinate plane is as follows: Where, d h The distance from the camera to the XOY plane of the object coordinate system is calculated based on the horizontal moiré pattern characteristics; d v f is the distance from the camera to the XOY plane of the object coordinate system, calculated based on the vertical moiré pattern features; f is the focal length of the camera. code f represents the spatial frequency magnitude of the visual markers in Moore's Law. c θ represents the spatial frequency of the camera color filter array CFA. c The spatial frequency direction of the CFA in the object coordinate system; The calculated distance d h and d v The mean value d is calculated to enhance the object's inherent Z-axis vibration mode while eliminating random noise. The calculation method is as follows: The distance d corresponding to each frame of the image is obtained by iterating through the images. The distance d0 corresponding to the initial frame is then subtracted to obtain the Z-axis vibration curve z in the object coordinate system OXYZ. The above calculation results x, y, and z are the X-axis, Y-axis, and Z-axis vibration curves obtained from one sub-region of interest. The above steps are repeated for each of the four sub-regions of interest, and the triaxial vibration curves obtained from the i-th region of interest are denoted as x, y, and z, respectively. i y i and z i , where i = 1, 2, 3, 4.

7. The mechanical vibration detection system based on moiré patterns according to claim 6, characterized in that, The multi-channel fusion algorithm of the vibration curve calculation module is as follows: The arithmetic mean of the data obtained from the four sub-regions of interest is calculated to obtain the final triaxial vibration curves x′, y′, and z′ of the object, which are as follows:

8. The mechanical vibration detection system based on moiré patterns according to claim 7, characterized in that, The method for calculating the vibration amplitude in the vibration parameter estimation module is as follows: The X-axis vibration amplitude A is calculated based on the X-axis vibration curve x′. x The calculation method is as follows: x normal =x′-mean(x′) Where mean(x′) represents the result of calculating the mean of the time series data x′; x normal The normalized data for x′; For time series data x normal The i-th data in; rms x This represents the effective value of the amplitude. The Y-axis vibration amplitude A is determined based on the Y-axis vibration curve y′. y The calculation method is as follows: y normal =y′-mean(y′) Where mean(y′) represents the result of calculating the mean of the time series data y′; normal The normalized data for y′; For time series data y normal The i-th data in; rms y This represents the effective value of the amplitude. The Z-axis vibration amplitude A is calculated based on the Z-axis vibration curve z′. z The calculation method is as follows: z normal =z′-mean(z′) Where mean(z′) represents the result of calculating the mean of the time series data z′; z normal For the normalized data of z′; For time series data z normal The i-th data in; rms z This represents the effective value of the amplitude. The method for calculating the vibration frequency in the vibration parameter estimation module is as follows: The largest prime number within the adjustable frame rate range of the camera is selected as the first frame rate fps1, and the corresponding triaxial vibration curve x is obtained. fps1 y fps1 and z fps1 For the triaxial vibration curve x fps1 y fps1 and z fps1 Perform fast Fourier transforms on each to obtain the amplitude spectrum X. fps1 ,Y fps1 Z fps1 , X amplitude spectrum fps1 ,Y fps1 Z fps1 The frequencies corresponding to the points with the largest amplitude values ​​are denoted as the observation frequencies obs. x1 obx y1 obs z1 Three observation frequencies (obs) x1 obs y1 obs z1 The mean is denoted as obs1; Let all possible true frequencies at the observed frequency obs1 be denoted as... Where frequency max Set to the maximum actual frequency that may occur in the application scenario, b i The calculation formula is as follows: Where b1 = obs1; b2 = fps1 - obs1; If we set the maximum deviation between the ideal and actual values ​​of the true frequency to ε, then the possible true frequency ranges at the observed frequency obs1 are: The second frame rate, fps2, is selected based on a dual frame rate selection algorithm, which is as follows: The second frame rate (fps2) is in the set. Choose from the options in set A, and for each frame rate, calculate the possible true frequency ranges. All observation frequency ranges at this frame rate The choice makes The frame rate with the largest minimum interval in the interval is taken as fps2; Based on the second frame rate fps2, the corresponding triaxial vibration curve x is obtained. fps2 y fps2 and z fps2 For the triaxial vibration curve x fps2 y fps2 and z fps2 Perform fast Fourier transforms on each to obtain the amplitude spectrum X. fps2 ,Y fps2 Z fps2 , X amplitude spectrum fps2 ,Y fps2 Z fps2 The frequencies corresponding to the points with the largest amplitude values ​​are denoted as the observation frequencies obs. x2 obs y2 obs z2 Three observation frequencies (obs) x2 obs y2 obs z2 The mean is denoted as obs2; The vibration frequency was calculated based on the dual-frame-rate frequency reconstruction method, which is as follows: Based on the relationship between the actual frequency, the observed frequency, and the camera frame rate, within the maximum frequency range [0, frequency]... max Find all possible true frequencies corresponding to the observed frequencies obs1 and obs2 within the range, and denote them as follows: and True frequency F t Observation frequency F o The relationship between the camera frame rate (fps) and the camera frame rate is as follows: Where MOD(·) is the remainder function; When there exist i and j such that a i and c j Satisfying relation |a i -c j When |<ε, we consider a i and c j They are equal, where ε is the maximum deviation between the ideal and actual values ​​of the true frequency; the formula for calculating the vibration frequency F is as follows:

9. A method for detecting mechanical vibration based on moiré patterns, based on the system described in any one of claims 1-8, characterized in that, The method includes the following steps: 1) Use a camera to capture video of a target object with moiré visual markers pasted on its surface at a first frame rate of fps1. The video frames contain moiré patterns with low spatial frequency. 2) Based on the ArUco markers, the regions of interest in the video frame sequence are coarsely selected and divided into 4 sub-regions of interest. The size of the regions of interest is then finely adjusted based on the robustness of the moiré features. 3) Perform image enhancement and binarization operations on the video frame sequence; 4) For each sub-region of interest, moiré features are extracted for each frame of the video using a lightweight spectral weighted feature extraction algorithm. The vibration curves of the X and Y axes are calculated using an in-plane vibration model based on the phase of the moiré pattern. The vibration curve of the Z axis is calculated using an out-of-plane vibration model based on the spatial frequency of the moiré pattern. 5) The data from the four sub-regions of interest are fused based on a multi-channel fusion algorithm to obtain the fused triaxial vibration curve; 6) Estimate the triaxial vibration amplitude based on the triaxial vibration curves obtained in step 5); 7) Select the second frame rate fps2 based on the dual frame rate selection algorithm for video capture, and repeat steps 2) to 5); 8) Estimating vibration frequency based on dual frame rate frequency reconstruction method.

Images