Mechanical vibration detection system and method based on moire
Through the contactless vibration detection system based on the camera, the molar feature and multi-channel fusion algorithm are used to solve the problems of insufficient accuracy, high cost and poor robustness of mechanical vibration detection, and high-precision and low-cost micron-level vibration detection are achieved. Especially effective detection in high-frequency vibration scenarios.
Patent Information
- Application Number
- CN202411012068.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-26
AI Technical Summary
The existing mechanical vibration detection methods have problems such as insufficient accuracy, high cost and poor robustness, and are particularly difficult to achieve micron-level mechanical vibration detection.
The contactless vibration detection system based on the camera is adopted, and the three-axis vibration curve calculation module and vibration parameter estimation module are used to calculate the three-axis vibration curve of the target object through the image preprocessing module, the vibration curve calculation module and the vibration parameter estimation module. The molar phase and spatial frequency characteristics are used, combined with the multi-channel fusion algorithm, and the three-axis vibration curve of the target object are calculated and Vibration parameters to achieve high-precision micron-scale vibration detection.
It realizes ultra-high-precision micron-scale vibration detection, which is robust and low in cost, and breaks through the limitations of traditional methods in high-frequency vibration detection, and the minimum vibration amplitude that can be perceived reaches 2um.
Smart Images

Figure CN118960933A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of vibration detection, and in particular relates to a mechanical vibration detection system and method based on moiré patterns. Background Art
[0002] Micro-vibrations are common in both daily life and industrial environments, and their detection and evaluation are crucial. In the case of industrial machinery, sudden changes in vibration direction, amplitude or frequency may indicate different faults inside the equipment; long-term micro-vibrations of building structures may mean potential structural damage and may even affect the health of residents. Traditional vibration sensors, such as piezoelectric accelerometers or eddy current displacement sensors, are usually attached directly to the target object, which may introduce potential resonance or change the inherent vibration state of the target object. In addition, researchers have explored solutions based on technologies such as laser, RFID, and millimeter waves; however, laser devices usually only allow single-point detection and require extremely high costs to ensure both detection distance and accuracy, while radio frequency signals have to overcome complex multipath interference and have limited ability to detect micro-vibration amplitudes. Therefore, there is an urgent need for a lightweight, high-precision, low-cost, and robust vibration detection solution to achieve micron-level vibration detection.
[0003] Existing mechanical vibration detection methods mainly include methods based on specific sensors, methods based on radio frequency signals, and methods based on traditional computer vision;
[0004] Methods based on specific sensors include contact and non-contact. Contact detection methods include accelerometers, displacement sensors, piezoelectric sensors, etc., which are usually directly installed on the target object for detection. However, adding additional components to the target object may change its vibration characteristics, which may affect the normal operation of the equipment. Non-contact detection methods mainly refer to laser-based methods. High-end equipment can achieve high-precision detection of micro-vibrations over long distances, but the high price hinders its large-scale application in actual scenarios.
[0005] Vibration detection based on radio frequency signals mainly includes two categories: RFID-based methods and millimeter wave-based methods. RFID-based methods detect vibrations by sensing subtle changes in the backscattered radio frequency signals modulated by the vibration signal through the tag, but RFID-based solutions show limited amplitude perception capabilities, making it challenging to sense micro-vibrations with amplitudes below 0.1 mm in most mechanical vibration scenarios. Compared with RFID signals, millimeter waves have the advantage of millimeter-level wavelengths, making millimeter wave-based methods more sensitive to micro-vibrations. However, millimeter wave-based solutions are severely affected by multipath effects in actual industrial environments.
[0006] Vision-based vibration detection methods mainly include marker-based methods and marker-free methods; marker-based methods require custom markers to be attached to the target object; although these methods can achieve sub-pixel perception through certain vibration amplification techniques, they still cannot achieve micron-level mechanical vibration detection. Marker-free methods use the inherent features of objects, such as corners, edges, and textures, to sense the vibration of objects. However, these methods rely too much on the surface features of the target itself, resulting in significant differences in detection performance in different scenarios. In addition, an important limitation of traditional computer vision-based methods is that they cannot break through the frame rate limit of the camera to achieve high-frequency detection. In recent years, some researchers have used customized devices consisting of a dual-shutter camera, a beam splitter, an objective lens, and a cylindrical lens to achieve high-frequency vibration detection. However, the complexity and high computational requirements of customized devices have greatly hindered their availability and practicality. Summary of the invention
[0007] In view of the above-mentioned deficiencies in the prior art, an object of the present invention is to provide a mechanical vibration detection system and method based on moiré to solve the problems of the existing mechanical vibration detection methods such as insufficient accuracy, high cost, poor robustness, etc.
[0008] To achieve the above object, the technical solution adopted by the present invention is as follows:
[0009] A mechanical vibration detection system based on moiré patterns of the present invention is applied to a computing device connected to a camera in a non-contact vibration detection scenario based on a camera, and includes: an image preprocessing module, a vibration curve calculation module and a vibration parameter estimation module;
[0010] The image preprocessing module is used to select the region of interest from the image captured by the camera, perform image enhancement and convert it into a binary image;
[0011] A vibration curve calculation module is used to solve the three-axis vibration curve of the target object to be detected in the image in the object coordinate system OXYZ within a time interval;
[0012] The vibration parameter estimation module is used to estimate the vibration amplitude and vibration frequency of the target object to be detected in the image.
[0013] Furthermore, the image preprocessing module roughly selects the region of interest according to the ArUco markers at the four corner points of the molar vision marker, divides the region of interest into four sub-regions of interest with a certain overlap, and fine-tunes each sub-region of interest according to the molar feature robustness criterion to improve the robustness of the entire system. For each region of interest in each frame of the image, image enhancement and binarization are performed.
[0014] Furthermore, the process of roughly selecting and dividing the region of interest in the image preprocessing module is specifically as follows:
[0015] The image captured by the camera is roughly selected based on the ArUco markers for the region of interest, with a side length of L;
[0016] The edges of the roughly selected region of interest with a side length of L are cropped to obtain a central region of interest with a side length of 0.8L to eliminate the influence of edge unstable features on system accuracy;
[0017] The central region of interest with a side length of 0.8L is divided into four overlapping sub-regions of interest with a side length of 0.5L.
[0018] Furthermore, the sub-ROI fine-tuning process in the image preprocessing module is specifically as follows:
[0019] For the current sub-region of interest, select the first 10 frames of the video captured by the camera, and calculate the spatial frequency vector f according to the vibration curve calculation module m , to calculate the average value of all horizontal normalized frequency components u and vertical normalized frequency components v, recorded as and
[0020] like The robustness of the molar feature is defined as: in, Indicates distance The nearest integer; traverse the length of the sub-ROI in the range of [max(1,0.5L-20), 0.5L], and select the length that maximizes the robustness of the molar feature as the final length of the sub-ROI, denoted by L o ; The width and height of the sub-region of interest after fine-tuning are L u =L o and L v =L o ;
[0021] like The robustness of the molar feature is defined as: in, Indicates distance The nearest integer; traverse the length of the sub-ROI in the range of [max(1,0.5L-20), 0.5L], and select the length that maximizes the robustness of the molar feature as the final length of the sub-ROI, denoted by L o ; The width and height of the sub-region of interest after fine-tuning are L u =L o and L v =L o ;
[0022] like According to the robustness of the molar feature Fine-tune the width of the sub-ROI in the range [max(1,0.5L-20), 0.5L] according to the robustness of the moiré feature. Fine-tune the height of the region of interest in the range of [max(1,0.5L-20), 0.5L]. After fine-tuning, the width and height of the region of interest are L respectively. u and L v .
[0023] Furthermore, the image enhancement and binarization process in the image preprocessing module is specifically as follows:
[0024] Perform histogram equalization on the extracted region of interest to enhance the contrast;
[0025] Perform Gaussian filtering on the equalized image to smooth out random noise;
[0026] Perform adaptive binarization on the Gaussian filtered image.
[0027] Furthermore, the vibration curve calculation module extracts moiré features for each sub-region of interest in each frame of image by using a spectrum weighted feature extraction algorithm: the spatial frequency vector f m (including: spatial frequency size f m and the spatial frequency direction θ m ) and phase The vibration curves of the X-axis and Y-axis are then calculated using an in-plane vibration model based on the moiré phase, and the vibration curve of the Z-axis is calculated using an out-of-plane vibration model based on the moiré spatial frequency. The data of multiple sub-regions of interest are fused using a multi-channel fusion algorithm to obtain the final X-axis vibration curve x′, Y-axis vibration curve y′, and Z-axis vibration curve z′.
[0028] Furthermore, the origin O in the object coordinate system OXYZ is the geometric center of the Moore vision marker in the initial frame; the X axis and Y axis of the object coordinate system OXYZ are respectively parallel to the long side and the short side of the Moore vision marker, and the Z axis is orthogonal to the XOY plane.
[0029] Furthermore, the moiré vision mark refers to two groups of mutually perpendicular one-dimensional stripes with the same spatial frequency printed on the paper; the directions of the two groups of one-dimensional stripes are parallel to the X-axis and Y-axis of the object coordinate system OXYZ by default.
[0030] Furthermore, the process of extracting molar features in the vibration curve calculation module is specifically as follows:
[0031] Perform fast Fourier transform on the image after image preprocessing to obtain a spectrum diagram containing moiré features;
[0032] Find the point with the largest amplitude in the [-45°, 45°) region of the spectrum graph, and mark its coordinates as (u v ,v v ), take this point as the upper left corner, upper right corner, lower right corner, and lower left corner of the 2×2 area, calculate the amplitude sum of the four 2×2 areas respectively, select the 2×2 area with the largest amplitude sum as the final weighted area, use the amplitude value as the weight, and calculate the weighted coordinates of the brightest pulse point (u v_w ,v v_w );
[0033] Find the point with the largest amplitude in the [45°, 135°] region of the spectrum graph, and mark its coordinates as (u h ,v h ), take this point as the upper left corner, upper right corner, lower right corner, and lower left corner of the 2×2 area, calculate the amplitude sum of the four 2×2 areas respectively, select the 2×2 area with the largest amplitude sum as the final weighted area, use the amplitude value as the weight, and calculate the weighted coordinates of the brightest pulse point (u h_w ,v h_w );
[0034] According to the weighted coordinates of the two brightest pulse points extracted above (u v_w ,v v_w ) and (u h_w ,v h_w ), the calculation method of the moiré feature in the camera coordinate system is as follows:
[0035]
[0036] Among them, f h and θ h Indicates the spatial frequency and direction of horizontal moiré; f v and θ v Indicates the spatial frequency and direction of vertical moiré; σ u and σ v The conversion coefficient L is used to convert the normalized frequencies u and v in the spectrum diagram into absolute frequencies with real physical meanings. c Indicates the pixel unit size of CFA; L u and L v Respectively represent the width and height of the region of interest in pixels;
[0037] Calculate the first 10 frames of image extraction (u h ,v h ) and (u v ,v v ), denoted as (u h_m ,v h_m ) and (uv_m ,v v_m ), extract the (u h_m ,v h_m ) and (u v_m ,v v_m ) coordinates as the phase of the entire image, denoted by and
[0038] and Denotes two sets of calculated mole features, which are respectively recorded as horizontal mole features and vertical mole features.
[0039] Furthermore, the in-plane vibration model based on the moiré phase in the vibration curve calculation module is:
[0040] The displacements Δx and Δy of the object on the X-axis and Y-axis between adjacent frames are calculated in sequence. The calculation method is as follows:
[0041]
[0042] Among them, T code represents the size of the fringe space period of the moiré visual marker, Indicates phase The difference between adjacent frames, Indicates phase The difference between adjacent frames;
[0043] The X-axis coordinate and Y-axis coordinate of the object in the initial frame are set to 0. The X-axis coordinate of the next frame is equal to the sum of the X-axis coordinate of the previous frame and the X-axis displacement Δx, and the Y-axis coordinate of the next frame is equal to the sum of the Y-axis coordinate of the previous frame and the Y-axis displacement Δy. The X-axis vibration curve x and the Y-axis vibration curve y of the object are calculated respectively.
[0044] Furthermore, the off-plane vibration model based on the moiré spatial frequency in the vibration curve calculation module is:
[0045] The distance calculation formula from the camera imaging plane to the XOY plane of the object coordinate system is as follows:
[0046]
[0047] Among them, d h is the distance from the camera to the XOY plane of the object coordinate system calculated based on the horizontal moiré feature; d v is the distance from the camera to the XOY plane of the object coordinate system calculated based on the vertical moiré feature; f is the focal length of the camera, f code is the spatial frequency of the molar visual marker, f c is the spatial frequency of the camera color filter array CFA, θc is the spatial frequency direction of CFA in the object coordinate system;
[0048] The calculated distance d h and d v Find the mean value d, which enhances the object's inherent Z-axis vibration mode while eliminating random noise. The calculation method is as follows:
[0049]
[0050] The distance d corresponding to each frame image is obtained by traversing and solving, and the distance d0 corresponding to the initial frame is subtracted to obtain the Z-axis vibration curve z in the object coordinate system OXYZ;
[0051] The above calculation results x, y, z are the X-axis, Y-axis and Z-axis vibration curves calculated for one sub-region of interest. Repeat the above steps for the four sub-regions of interest respectively, and the three-axis vibration curves calculated from the i-th region of interest are recorded as x, y, and z. i ,y i and z i , where i=1,2,3,4.
[0052] Furthermore, the multi-channel fusion algorithm of the vibration curve calculation module is specifically as follows:
[0053] The data calculated from the four sub-regions of interest are averaged to obtain the final three-axis vibration curves x′, y′, and z′ of the object:
[0054]
[0055] Furthermore, the vibration amplitude is calculated in the vibration parameter estimation module as follows:
[0056] Determine the X-axis vibration amplitude A based on the X-axis vibration curve x′ x , the calculation method is as follows:
[0057] x normal =x′-mean(x′)
[0058]
[0059] Where, mean(x′) represents the result of calculating the mean of the time series data x′; x normal is the normalized data of x′; is the time series data x normal The i-th data in rms x is the effective value of amplitude;
[0060] Determine the Y-axis vibration amplitude A based on the Y-axis vibration curve y′ y , the calculation method is as follows:
[0061] y normal =y′-mean(y′)
[0062]
[0063] Where, mean(y′) represents the result of calculating the mean of the time series data y′; y normal is the normalized data of y′; is the time series data y normal The i-th data in rms y is the effective value of amplitude;
[0064] Determine the Z-axis vibration amplitude A based on the Z-axis vibration curve z′ z , the calculation method is as follows:
[0065] z normal = z′-mean(z′)
[0066]
[0067] Where, mean(z′) represents the result of calculating the mean of the time series data z′; z normal is the normalized data of z′; is the time series data z normal The i-th data in rms z is the effective value of the amplitude.
[0068] Furthermore, the vibration frequency is calculated in the vibration parameter estimation module as follows:
[0069] Select the largest prime number within the adjustable frame rate range of the camera as the first frame rate fps1, and get the corresponding three-axis vibration curve x fps1 ,y fps1 and z fps1 , for the triaxial vibration curve x fps1 ,y fps1和 z fps1 Perform fast Fourier transform respectively to get the amplitude spectrum X fps1 ,Y fps1 ,Z fps1 , the amplitude spectrum X fps1 ,Y fps1 ,Z fps1 The frequencies corresponding to the points with the largest amplitude values are recorded as observation frequencies obs x1 , obs y1 , obs z1 ; Three observation frequencies obs x1 , obs y1 , obs z1 The mean of is recorded as obs1;
[0070] All possible true frequencies under the observed frequency obs1 are recorded as where frequency max Set to the maximum real frequency that may occur in the application scenario, b i The calculation formula is as follows:
[0071]
[0072] Among them, b1=obs1; b2=fps1-obs1;
[0073] Assuming the maximum deviation between the ideal value and the actual value of the true frequency is ε, then all possible true frequency intervals under the observed frequency obs1 are
[0074] The second frame rate fps2 is selected according to the dual frame rate selection algorithm, and the dual frame rate selection algorithm is as follows:
[0075] The second frame rate fps2 is in the set For each frame rate in set A, calculate the possible real frequency interval All observed frequency intervals at this frame rate Select so that The frame rate with the smallest interval and the largest interval in the interval is taken as fps2;
[0076] Based on the second frame rate fps2, the corresponding three-axis vibration curve x is obtained. fps2 ,y fps2 and z fps2 , for the triaxial vibration curve x fps2 ,y fps2 and z fps2 Perform fast Fourier transform respectively to get the amplitude spectrum X fps2 ,Y fps2 ,Z fps2 , the amplitude spectrum X fps2 ,Y fps2 ,Z fps2 The frequencies corresponding to the points with the largest amplitude values are recorded as observation frequencies obs x2 , obs y2 , obs z2 ; Three observation frequencies obsx2, obs y2 , obs z2 The mean of is recorded as obs2;
[0077] The vibration frequency is calculated based on the dual frame rate frequency reconstruction method, and the dual frame rate frequency reconstruction method is as follows:
[0078] According to the relationship between the real frequency, the observed frequency and the camera frame rate, in the maximum frequency range [0,frequency max ] to find all possible true frequencies corresponding to the observed frequencies obs1 and obs2, respectively and True frequency F t , observation frequency F o The relationship between the camera frame rate fps is as follows:
[0079]
[0080] Among them, MOD(·) is the remainder function;
[0081] When there exist i and j such that a i and c j Satisfaction relationship |a i -c j |<ε, it is considered that a i and c j equal, ε is the maximum deviation between the ideal value and the actual value of the true frequency; the calculation formula of the vibration frequency F is as follows:
[0082]
[0083] Mechanical vibration detection in the present invention refers to the use of a camera to detect the vibration state of a target object with a moiré visual marker attached; it is applicable to scenarios where the camera is fixed and the target object is moving. The present invention describes the vibration state of the target object as a three-axis vibration curve of the target object in the object coordinate system OXYZ: the X-axis vibration curve x′, the Y-axis vibration curve y′ and the Z-axis vibration curve z′, as well as vibration parameters: the three-axis vibration amplitude A x , A y , A z and vibration frequency F; the present invention can output the three-axis vibration curve, three-axis vibration amplitude and vibration frequency of the target object in the object coordinate system.
[0084] The present invention also provides a mechanical vibration detection method based on moiré patterns, based on the above system, comprising the following steps:
[0085] 1) using a camera to collect video of a target object with a moiré visual marker attached to its surface at a first frame rate fps1, wherein the video frame contains a moiré pattern image with a low spatial frequency;
[0086] 2) The region of interest in the video frame sequence is roughly selected according to the ArUco markers, divided into four sub-regions of interest, and the size of the region of interest is fine-tuned according to the robustness of the moiré feature;
[0087] 3) Perform image enhancement and binarization operations on the video frame sequence;
[0088] 4) For each sub-region of interest, a lightweight spectrum weighted feature extraction algorithm is used to extract moiré features for each frame of the video, and the vibration curves of the X-axis and Y-axis are calculated using an in-plane vibration model based on the moiré phase; the vibration curve of the Z-axis is calculated using an out-of-plane vibration model based on the moiré spatial frequency;
[0089] 5) Based on the multi-channel fusion algorithm, the data of the four sub-regions of interest are fused to obtain the fused three-axis vibration curve;
[0090] 6) estimating the three-axis vibration amplitudes respectively according to the three-axis vibration curves obtained in step 5);
[0091] 7) Select the second frame rate fps2 for video acquisition based on the dual frame rate selection algorithm, and repeat steps 2) to 5);
[0092] 8) Estimate vibration frequency based on dual frame rate frequency reconstruction method.
[0093] Beneficial effects of the present invention:
[0094] 1. Ultra-high precision: The moiré pattern formed by the projection superposition of the color filter array CFA in the camera and the moiré visual marker on the imaging plane is extremely sensitive to the relative posture change between the camera and the target object; the movement of the target object in the XOY plane of the object coordinate system will cause a significant change in the phase characteristics of the moiré pattern; the movement of the target object along the Z-axis direction in the object coordinate system will cause a significant change in the spatial frequency characteristics of the moiré pattern; based on the spectrum weighted feature extraction algorithm in the vibration curve calculation module, high-precision moiré features are extracted, and then through the in-plane vibration model based on the moiré phase, the out-of-plane vibration model based on the moiré spatial frequency and the multi-channel fusion algorithm, the amplitude perception accuracy of the system of the present invention can reach the micron level, and the minimum perceptible vibration amplitude is 2um.
[0095] 2. Robustness: Compared with traditional feature-based computer vision methods, the statistical characteristics of moiré splines in the frequency domain can provide more accurate and robust information for vibration detection, and the sub-region of interest fine-tuning in the image preprocessing module of the present invention can further improve the robustness of the system; while the number of feature points of traditional computer vision methods is limited and easily affected by the environment.
[0096] 3. Low cost: The present invention only requires an entry-level industrial camera and moire vision markers printed on ordinary paper to achieve high-precision mechanical vibration detection at a low cost.
[0097] 4. High-frequency vibration detection capability: The dual-frame rate frequency reconstruction and dual-frame rate selection method based on the vibration parameter restoration module of the present invention can break through the limitation of Nyquist sampling theorem and realize the detection of high-frequency vibrations up to 10 times the camera frame rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0098] Figure 1 is a structural diagram of a mechanical vibration detection system in an embodiment of the present invention;
[0099] Figure 2 is a schematic diagram of the molar visual marking in the present invention;
[0100] Figure 3 A schematic diagram of rough selection and division of the region of interest of an image in the present invention;
[0101] Figure 4 This is a flowchart of fine-tuning the image region of interest in the present invention;
[0102] Figure 5 This is a schematic diagram of the molar feature extraction principle in the present invention;
[0103] Figure 6 It is a flow chart of the in-plane vibration detection method based on moiré phase in the present invention;
[0104] Figure 7 It is a flow chart of the off-plane vibration detection method based on the moiré spatial frequency in the present invention;
[0105] Figure 8 It is the principle diagram of the multi-channel fusion algorithm in the present invention;
[0106] Fig. 9 This is a flow chart of the vibration amplitude calculation method in the present invention;
[0107] Fig.10 This is a flow chart of the vibration frequency calculation method in the present invention. DETAILED DESCRIPTION
[0108] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.
[0109] Reference Figure 1 to Figure 2 As shown, a mechanical vibration detection system based on moiré pattern of the present invention is applied to a computing device connected to a camera in a non-contact vibration detection scenario based on a camera, and includes: an image preprocessing module, a vibration curve calculation module and a vibration parameter estimation module;
[0110] The image preprocessing module is used to select a region of interest (ROI) from the image captured by the camera, perform image enhancement and convert it into a binary image;
[0111] Specifically, the image preprocessing module roughly selects the region of interest according to the ArUco marks (positioning marks) at the four corner points of the molar vision marker, divides the region of interest into four sub-regions of interest with a certain overlap, and fine-tunes each sub-region of interest according to the molar feature robustness criterion to improve the robustness of the entire system. For each region of interest in each frame image, image enhancement and binarization are performed.
[0112] Reference Figure 3 As shown, the process of roughly selecting and dividing the region of interest in the image preprocessing module is specifically as follows:
[0113] The image captured by the camera is roughly selected according to the ArUco mark of the region of interest, with a side length of L (unit: pixel);
[0114] The edges of the roughly selected region of interest with a side length of L are cropped to obtain a central region of interest with a side length of 0.8L to eliminate the influence of edge unstable features on system accuracy;
[0115] The central region of interest with a side length of 0.8L is divided into four overlapping sub-regions of interest with a side length of 0.5L.
[0116] Reference Figure 4 As shown, the sub-ROI fine-tuning process in the image preprocessing module is specifically as follows:
[0117] For the current sub-region of interest, select the first 10 frames of the video captured by the camera, and calculate the spatial frequency vector f according to the vibration curve calculation module m , to calculate the average value of all horizontal normalized frequency components u and vertical normalized frequency components v, recorded as and
[0118] like The robustness of the molar feature is defined as: in, Indicates distance The nearest integer; traverse the length of the sub-ROI in the range of [max(1,0.5L-20), 0.5L], and select the length that maximizes the robustness of the molar feature as the final length of the sub-ROI, denoted by L o ; The width and height of the sub-region of interest after fine-tuning are L u =L o and L v =L o ;
[0119] like The robustness of the molar feature is defined as: in, Indicates distance The nearest integer; traverse the length of the sub-ROI in the range of [max(1,0.5L-20), 0.5L], and select the length that maximizes the robustness of the molar feature as the final length of the sub-ROI, denoted by L o ; The width and height of the sub-region of interest after fine-tuning are L u =L o and L v =L o ;
[0120] like According to the robustness of the molar feature Fine-tune the width of the sub-ROI in the range [max(1,0.5L-20), 0.5L] according to the robustness of the moiré feature. Fine-tune the height of the region of interest in the range of [max(1,0.5L-20), 0.5L]. After fine-tuning, the width and height of the region of interest are L respectively. u and L v .
[0121] Specifically, the image enhancement and binarization process in the image preprocessing module is as follows:
[0122] Perform histogram equalization on the extracted region of interest to enhance the contrast;
[0123] Perform Gaussian filtering on the equalized image to smooth out random noise;
[0124] Perform adaptive binarization on the Gaussian filtered image.
[0125] A vibration curve calculation module is used to solve the three-axis vibration curve of the target object to be detected in the image in the object coordinate system OXYZ within a time interval;
[0126] Specifically, the vibration curve calculation module extracts moiré features for each sub-region of interest in each frame of image by using a spectrum weighted feature extraction algorithm: the spatial frequency vector f m (including: spatial frequency size f m and the spatial frequency direction θ m ) and phase The vibration curves of the X-axis and Y-axis are then calculated using an in-plane vibration model based on the moiré phase, and the vibration curve of the Z-axis is calculated using an out-of-plane vibration model based on the moiré spatial frequency. The data of multiple sub-regions of interest are fused using a multi-channel fusion algorithm to obtain the final X-axis vibration curve x′, Y-axis vibration curve y′, and Z-axis vibration curve z′.
[0127] Specifically, the origin O in the object coordinate system OXYZ is the geometric center of the Moore vision marker in the initial frame; the X axis and Y axis of the object coordinate system OXYZ are respectively parallel to the long side and the short side of the Moore vision marker, and the Z axis is orthogonal to the XOY plane.
[0128] Specifically, the moiré visual mark refers to two groups of mutually perpendicular one-dimensional stripes with the same spatial frequency printed on the paper; the directions of the two groups of one-dimensional stripes are parallel to the X-axis and the Y-axis of the object coordinate system OXYZ by default.
[0129] Reference Figure 5 As shown, the process of extracting molar features in the vibration curve calculation module is specifically as follows:
[0130] Perform fast Fourier transform on the image after image preprocessing to obtain a spectrum diagram containing moiré features;
[0131] Find the point with the largest amplitude in the [-45°, 45°) region of the spectrum graph, and mark its coordinates as (u v ,v v ), take this point as the upper left corner, upper right corner, lower right corner, and lower left corner of the 2×2 area, calculate the amplitude sum of the four 2×2 areas respectively, select the 2×2 area with the largest amplitude sum as the final weighted area, use the amplitude value as the weight, and calculate the weighted coordinates of the brightest pulse point (u v_w ,v v_w );
[0132] Find the point with the largest amplitude in the [45°, 135°] region of the spectrum graph, and mark its coordinates as (u h ,v h ), take this point as the upper left corner, upper right corner, lower right corner, and lower left corner of the 2×2 area, calculate the amplitude sum of the four 2×2 areas respectively, select the 2×2 area with the largest amplitude sum as the final weighted area, use the amplitude value as the weight, and calculate the weighted coordinates of the brightest pulse point (u h_w ,v h_w );
[0133] According to the weighted coordinates of the two brightest pulse points extracted above (u v_w ,v v_w ) and (u h_w ,v h_w ), the calculation method of the moiré feature in the camera coordinate system is as follows:
[0134]
[0135] Among them, f h and θ h Indicates the spatial frequency and direction of horizontal moiré; f v and θv Indicates the spatial frequency and direction of vertical moiré; σ u and σ v The conversion coefficient L is used to convert the normalized frequencies u and v in the spectrum diagram into absolute frequencies with real physical meanings. c Indicates the pixel unit size of CFA; L u and L v Respectively represent the width and height of the region of interest in pixels;
[0136] Calculate the first 10 frames of image extraction (u h ,v h ) and (u v ,v v ), denoted as (u h_m ,v h_m ) and (u v_m ,v v_m ), extract the (u h_m ,v h_m ) and (u v_m ,v v_m ) coordinates as the phase of the entire image, denoted by and
[0137] and Denotes two sets of calculated mole features, which are respectively recorded as horizontal mole features and vertical mole features.
[0138] Reference Figure 6 As shown, the in-plane vibration model based on the moiré phase in the vibration curve calculation module is:
[0139] The displacements Δx and Δy of the object on the X-axis and Y-axis between adjacent frames are calculated in sequence. The calculation method is as follows:
[0140]
[0141] Among them, T code represents the size of the fringe space period of the moiré visual marker, Indicates phase The difference between adjacent frames, Indicates phase The difference between adjacent frames;
[0142] The X-axis coordinate and Y-axis coordinate of the object in the initial frame are set to 0. The X-axis coordinate of the next frame is equal to the sum of the X-axis coordinate of the previous frame and the X-axis displacement Δx, and the Y-axis coordinate of the next frame is equal to the sum of the Y-axis coordinate of the previous frame and the Y-axis displacement Δy. The X-axis vibration curve x and the Y-axis vibration curve y of the object are calculated respectively.
[0143] Reference Figure 7 As shown, the off-plane vibration model based on the moiré spatial frequency in the vibration curve calculation module is:
[0144] The distance calculation formula from the camera imaging plane to the XOY plane of the object coordinate system is as follows:
[0145]
[0146] Among them, d h is the distance from the camera to the XOY plane of the object coordinate system calculated based on the horizontal moiré feature; d v is the distance from the camera to the XOY plane of the object coordinate system calculated based on the vertical moiré feature; f is the focal length of the camera, f code is the spatial frequency of the molar visual marker, f c is the spatial frequency of the camera color filter array CFA, θ c is the spatial frequency direction of CFA in the object coordinate system;
[0147] The calculated distance d h and d v Find the mean value d, which enhances the object's inherent Z-axis vibration mode while eliminating random noise. The calculation method is as follows:
[0148]
[0149] The distance d corresponding to each frame image is obtained by traversing and solving, and the distance d0 corresponding to the initial frame is subtracted to obtain the Z-axis vibration curve z in the object coordinate system OXYZ;
[0150] The above calculation results x, y, z are the X-axis, Y-axis and Z-axis vibration curves calculated for one sub-region of interest. Repeat the above steps for the four sub-regions of interest respectively, and the three-axis vibration curves calculated from the i-th region of interest are recorded as x, y, and z. i ,y i and z i , where i=1,2,3,4.
[0151] Reference Figure 8 As shown, the multi-channel fusion algorithm of the vibration curve calculation module is specifically as follows:
[0152] The data calculated from the four sub-regions of interest are averaged to obtain the final three-axis vibration curves x′, y′, and z′ of the object:
[0153]
[0154]
[0155] The vibration parameter estimation module is used to estimate the vibration amplitude and vibration frequency of the target object to be detected in the image.
[0156] Reference Fig. 9 As shown, the calculation method of the vibration amplitude in the vibration parameter estimation module is:
[0157] Determine the X-axis vibration amplitude A based on the X-axis vibration curve x′ x , the calculation method is as follows:
[0158] x normal =x′-mean(x′)
[0159]
[0160] Where, mean(x′) represents the result of calculating the mean of the time series data x′; x normal is the normalized data of x′; is the time series data x normal The i-th data in rms x is the effective value of amplitude;
[0161] Determine the Y-axis vibration amplitude A based on the Y-axis vibration curve y′ y , the calculation method is as follows:
[0162] y normal =y′-mean(y′)
[0163]
[0164] Where, mean(y′) represents the result of calculating the mean of the time series data y′; y normal is the normalized data of y′; is the time series data y normal The i-th data in rms y is the effective value of amplitude;
[0165] Determine the Z-axis vibration amplitude A based on the Z-axis vibration curve z′ z , the calculation method is as follows:
[0166] z normal = z′-mean(z′)
[0167]
[0168] Where, mean(z′) represents the result of calculating the mean of the time series data z′; z normal is the normalized data of z′; is the time series data z normal The i-th data in rms z is the effective value of the amplitude.
[0169] Reference Fig.10 As shown, the calculation method of the vibration frequency in the vibration parameter estimation module is:
[0170] Select the largest prime number within the adjustable frame rate range of the camera as the first frame rate fps1, and get the corresponding three-axis vibration curve x fps1 ,y fps1 and z fps1 , for the triaxial vibration curve x fps1 ,y fps1 and Z fps1 Perform fast Fourier transform respectively to get the amplitude spectrum X fps1 ,Y fps1 ,Z fps1 , the amplitude spectrum X fps1 ,Y fps1 ,Z fps1 The frequencies corresponding to the points with the largest amplitude values are recorded as observation frequencies obs x1 , obs y1 , obs z1 ; Three observation frequencies obs x1 , obs y1 , obs z1 The mean of is recorded as obs1;
[0171] All possible true frequencies under the observed frequency obs1 are recorded as where frequency max Set to the maximum real frequency that may occur in the application scenario, b i The calculation formula is as follows:
[0172]
[0173] Among them, b1=obs1; b2=fps1-obs1;
[0174] Assuming the maximum deviation between the ideal value and the actual value of the true frequency is ε, then all possible true frequency intervals under the observed frequency obs1 are
[0175] The second frame rate fps2 is selected according to the dual frame rate selection algorithm, and the dual frame rate selection algorithm is as follows:
[0176] The second frame rate fps2 is in the set For each frame rate in set A, calculate the possible real frequency interval All observed frequency intervals at this frame rate Select so that The frame rate with the smallest interval and the largest interval in the interval is taken as fps2;
[0177] Based on the second frame rate fps2, the corresponding three-axis vibration curve x is obtained. fps2 ,y fps2 and z fps2 , for the triaxial vibration curve x fps2 ,y fps2 and z fps2 Perform fast Fourier transform respectively to get the amplitude spectrum X fps2 ,Y fps2 ,Z fps2 , the amplitude spectrum X fps2 ,Y fps2 ,Z fps2 The frequencies corresponding to the points with the largest amplitude values are recorded as observation frequencies obs x2 , obs y2 , obs z2 ; Three observation frequencies obs x2 , obs y2 , obs z2 The mean of is recorded as obs2;
[0178] The vibration frequency is calculated based on the dual frame rate frequency reconstruction method, and the dual frame rate frequency reconstruction method is as follows:
[0179] According to the relationship between the real frequency, the observed frequency and the camera frame rate, in the maximum frequency range [0,frequency max ] to find all possible true frequencies corresponding to the observed frequencies obs1 and obs2, respectively and True frequency F t , observation frequency F o The relationship between the camera frame rate fps is as follows:
[0180]
[0181] Among them, MOD(·) is the remainder function;
[0182] When there exist i and j such that a i and c j Satisfaction relationship |a i -c j |<ε, it is considered that a i and c j equal, ε is the maximum deviation between the ideal value and the actual value of the true frequency; the calculation formula of the vibration frequency F is as follows:
[0183]
[0184] Mechanical vibration detection in the present invention refers to the use of a camera to detect the vibration state of a target object with a moiré visual marker attached; it is applicable to scenarios where the camera is fixed and the target object is moving. The present invention describes the vibration state of the target object as a three-axis vibration curve of the target object in the object coordinate system OXYZ: the X-axis vibration curve x′, the Y-axis vibration curve y′ and the Z-axis vibration curve z′, as well as vibration parameters: the three-axis vibration amplitude A x , A y , A z and vibration frequency F; the present invention can output the three-axis vibration curve, three-axis vibration amplitude and vibration frequency of the target object in the object coordinate system.
[0185] The present invention also provides a mechanical vibration detection method based on moiré patterns, based on the above system, comprising the following steps:
[0186] 1) Using a camera to collect video of a target object with a moiré visual mark attached to the surface at a first frame rate fps1 (the maximum prime number within the adjustable frame rate range of the camera), the video frame contains a moiré pattern image with a low spatial frequency;
[0187] 2) The region of interest in the video frame sequence is roughly selected according to the ArUco markers, divided into four sub-regions of interest, and the size of the region of interest is fine-tuned according to the robustness of the moiré feature;
[0188] 3) Perform image enhancement and binarization operations on the video frame sequence;
[0189] 4) For each sub-region of interest, a lightweight spectrum weighted feature extraction algorithm is used to extract moiré features for each frame of the video, and the vibration curves of the X-axis and Y-axis are calculated using an in-plane vibration model based on the moiré phase; the vibration curve of the Z-axis is calculated using an out-of-plane vibration model based on the moiré spatial frequency;
[0190] 5) Based on the multi-channel fusion algorithm, the data of the four sub-regions of interest are fused to obtain the fused three-axis vibration curve;
[0191] 6) estimating the three-axis vibration amplitudes respectively according to the three-axis vibration curves obtained in step 5);
[0192] 7) Select the second frame rate fps2 for video acquisition based on the dual frame rate selection algorithm, and repeat steps 2) to 5);
[0193] 8) Estimate vibration frequency based on dual frame rate frequency reconstruction method.
[0194] The present invention has many specific application paths. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principle of the present invention. These improvements should also be regarded as the protection scope of the present invention.
Claims
1. A mechanical vibration detection system based on moiré, applied to a computing device connected to a camera in a non-contact vibration detection scenario based on a camera, characterized in that: include: Image preprocessing module, vibration curve calculation module and vibration parameter estimation module; The image preprocessing module is used to select the region of interest from the image captured by the camera, perform image enhancement and convert it into a binary image; A vibration curve calculation module is used to solve the three-axis vibration curve of the target object to be detected in the image in the object coordinate system OXYZ within a time interval; The vibration parameter estimation module is used to estimate the vibration amplitude and vibration frequency of the target object to be detected in the image.
2. The mechanical vibration detection system based on moiré pattern according to claim 1, characterized in that: The image preprocessing module roughly selects the region of interest according to the ArUco markers at the four corner points of the molar vision marker, divides the region of interest into four sub-regions of interest with a certain overlap, and fine-tunes each sub-region of interest according to the molar feature robustness criterion to improve the robustness of the entire system. For each region of interest in each frame image, image enhancement and binarization are performed.
3. The mechanical vibration detection system based on moiré pattern according to claim 2, characterized in that: The process of roughly selecting and dividing the region of interest in the image preprocessing module is specifically as follows: The image captured by the camera is roughly selected based on the ArUco markers for the region of interest, with a side length of L; The edges of the roughly selected region of interest with a side length of L are cropped to obtain a central region of interest with a side length of 0.8L to eliminate the influence of edge unstable features on system accuracy; The central region of interest with a side length of 0.8L is divided into four overlapping sub-regions of interest with a side length of 0.5L.
4. The mechanical vibration detection system based on moiré pattern according to claim 3, characterized in that: The sub-ROI fine-tuning process in the image preprocessing module is specifically as follows: For the current sub-region of interest, select the first 10 frames of the video captured by the camera, and calculate the spatial frequency vector f according to the vibration curve calculation module m , to calculate the average value of all horizontal normalized frequency components u and vertical normalized frequency components v, recorded as and like The robustness of the molar feature is defined as: in, Indicates distance The nearest integer; traverse the length of the sub-ROI in the range of [max(1,0.5L-20), 0.5L], and select the length that maximizes the robustness of the molar feature as the final sub-ROI side length, denoted by L o ; The width and height of the sub-region of interest after fine-tuning are L u =L o and L v =L o ; like The robustness of the molar feature is defined as: in, Indicates distance The nearest integer; traverse the length of the sub-ROI in the range of [max(1,0.5L-20), 0.5L], and select the length that maximizes the robustness of the molar feature as the final sub-ROI side length, denoted by L o ; The width and height of the sub-region of interest after fine-tuning are L u =L o and L v =L o ; like According to the robustness of the molar feature Fine-tune the width of the sub-ROI in the range [max(1,0.5L-20), 0.5L] according to the robustness of the moiré feature. Fine-tune the height of the region of interest in the range of [max(1,0.5L-20), 0.5L]. After fine-tuning, the width and height of the region of interest are L respectively. u and L v .
5. The mechanical vibration detection system based on moiré pattern according to claim 1, characterized in that: The process of extracting molar features in the vibration curve calculation module is specifically as follows: Perform fast Fourier transform on the image after image preprocessing to obtain a spectrum diagram containing moiré features; Find the point with the largest amplitude in the [-45°, 45°) region of the spectrum graph, and mark its coordinates as (u v ,v v ), take this point as the upper left corner, upper right corner, lower right corner, and lower left corner of the 2×2 area, calculate the amplitude sum of the four 2×2 areas respectively, select the 2×2 area with the largest amplitude sum as the final weighted area, use the amplitude value as the weight, and calculate the weighted coordinates of the brightest pulse point (v v_w ,v v_w ); Find the point with the largest amplitude in the [45°, 135°] region of the spectrum graph, and mark its coordinates as (u h ,v h ), take this point as the upper left corner, upper right corner, lower right corner, and lower left corner of the 2×2 area, calculate the amplitude sum of the four 2×2 areas respectively, select the 2×2 area with the largest amplitude sum as the final weighted area, use the amplitude value as the weight, and calculate the weighted coordinates of the brightest pulse point (u h_w ,v h_w ); According to the weighted coordinates of the two brightest pulse points extracted above (u v_w ,v v_w ) and (u h_w ,v h_w ), the calculation method of moiré feature in the camera coordinate system is as follows: Among them, f h and θ h Indicates the spatial frequency and direction of horizontal moiré; f v and θ v Indicates the spatial frequency and direction of vertical moiré; σ u and σ v The conversion coefficient L is used to convert the normalized frequencies u and v in the spectrum diagram into absolute frequencies with real physical meanings. c Indicates the pixel unit size of CFA; L u and L v Respectively represent the width and height of the region of interest in pixels; Calculate the first 10 frames of image extraction (u h ,v h ) and (u v ,v v ), denoted as (u h_m ,v h_m ) and (u v_m ,v v_m ), extract the (u h_m ,v h_m ) and (u v_m ,v v_m ) coordinates as the phase of the entire image, denoted by and and Denotes two sets of calculated mole features, which are respectively recorded as horizontal mole features and vertical mole features.
6. The mechanical vibration detection system based on moiré pattern according to claim 5, characterized in that: The in-plane vibration model based on the moiré phase in the vibration curve calculation module is: The displacements Δx and Δy of the object on the X-axis and Y-axis between adjacent frames are calculated in sequence. The calculation method is as follows: Among them, T cofe represents the size of the fringe space period of the moiré visual marker, Indicates phase The difference between adjacent frames, Indicates phase The difference between adjacent frames; The X-axis coordinate and Y-axis coordinate of the object in the initial frame are set to 0. The X-axis coordinate of the next frame is equal to the sum of the X-axis coordinate of the previous frame and the X-axis displacement Δx, and the Y-axis coordinate of the next frame is equal to the sum of the Y-axis coordinate of the previous frame and the Y-axis displacement Δy. The X-axis vibration curve x and the Y-axis vibration curve y of the object are calculated respectively.
7. The mechanical vibration detection system based on moiré pattern according to claim 6, characterized in that: The off-plane vibration model based on the moiré spatial frequency in the vibration curve calculation module is: The distance calculation formula from the camera imaging plane to the XOY plane of the object coordinate system is as follows: Among them, d h is the distance from the camera to the XOY plane of the object coordinate system calculated based on the horizontal moiré feature; d v is the distance from the camera to the XOY plane of the object coordinate system calculated based on the vertical moiré feature; f is the focal length of the camera, f code is the spatial frequency of the molar visual marker, f c is the spatial frequency of the camera color filter array CFA, θ c is the spatial frequency direction of CFA in the object coordinate system; The calculated distance d h and d v Find the mean value d, which enhances the object's inherent Z-axis vibration mode while eliminating random noise. The calculation method is as follows: The distance d corresponding to each frame image is obtained by traversing and solving, and the distance d0 corresponding to the initial frame is subtracted to obtain the Z-axis vibration curve z in the object coordinate system OXYZ; The above calculation results x, y, z are the X-axis, Y-axis and Z-axis vibration curves calculated for one sub-region of interest. Repeat the above steps for the four sub-regions of interest respectively, and the three-axis vibration curves calculated from the i-th region of interest are recorded as x, y, and z. i ,y i and z i , where i=1,2,3,4.
8. The mechanical vibration detection system based on moiré pattern according to claim 7, characterized in that: The multi-channel fusion algorithm of the vibration curve calculation module is specifically as follows: The data calculated from the four sub-regions of interest are averaged to obtain the final three-axis vibration curves x′, y′, and z′ of the object:
9. The mechanical vibration detection system based on moiré pattern according to claim 8, characterized in that: The calculation method of the vibration amplitude in the vibration parameter estimation module is: Determine the X-axis vibration amplitude A based on the X-axis vibration curve x′ x , the calculation method is as follows: x normal =x′-mean(x′) Where, mean(x′) represents the result of calculating the mean of the time series data x′; x normal is the normalized data of x′; is the time series data x normal The i-th data in rms x is the effective value of amplitude; Determine the Y-axis vibration amplitude A based on the Y-axis vibration curve y′ y , the calculation method is as follows: y normal =y′-mean(y′) Where, mean(y′) represents the result of calculating the mean of the time series data y′; y normal is the normalized data of y′; is the time series data y normal The i-th data in rms y is the effective value of amplitude; Determine the Z-axis vibration amplitude A based on the Z-axis vibration curve z′ z , the calculation method is as follows: z normaol =z′-mean(z′) Where, mean(z′) represents the result of calculating the mean of the time series data z′; z normal is the normalized data of z′; is the time series data z normal The i-th data in rms z is the effective value of amplitude; The calculation method of the vibration frequency in the vibration parameter estimation module is: Select the largest prime number within the adjustable frame rate range of the camera as the first frame rate fps1, and get the corresponding three-axis vibration curve x fps1 ,y fps1 and z fps1 , for the triaxial vibration curve x fps1 ,y fps1 and f ps1 Perform fast Fourier transform respectively to get the amplitude spectrum X fps1 ,Y fps1 ,Z fps1 , the amplitude spectrum X fps1 ,Y fps1 ,Z fps1 The frequencies corresponding to the points with the largest amplitude values are recorded as observation frequencies obs x1 , obs y1 , obs z1 ; Three observation frequencies obs x1 , obs y1 , obs z1 The mean of is recorded as obs1; All possible true frequencies under the observed frequency obs1 are recorded as where frequency max Set to the maximum real frequency that may occur in the application scenario, b i The calculation formula is as follows: Among them, b1=obs1; b2=fps1-obs1; Assuming the maximum deviation between the ideal value and the actual value of the true frequency is ε, then all possible true frequency intervals under the observed frequency obs1 are The second frame rate fps2 is selected according to the dual frame rate selection algorithm, and the dual frame rate selection algorithm is as follows: The second frame rate fps2 is in the set For each frame rate in set A, calculate the possible real frequency interval All observed frequency intervals at this frame rate Select so that The frame rate with the smallest interval and the largest interval in the interval is taken as fps2; Based on the second frame rate fps2, the corresponding three-axis vibration curve x is obtained. fps2 ,y fps2 and z fps2 , for the triaxial vibration curve x fps2 ,y fps2 and z fps2 Perform fast Fourier transform respectively to get the amplitude spectrum X fps2 ,Y fps2 ,Z fps2 , the amplitude spectrum X fps2 ,Y fps2 ,Z fps2 The frequencies corresponding to the points with the largest amplitude values are recorded as observation frequencies obs x2 , obs y2 , obs z2 ; Three observation frequencies obs x2 , obs y2 , obs z2 The mean of is recorded as obs2; The vibration frequency is calculated based on the dual frame rate frequency reconstruction method, and the dual frame rate frequency reconstruction method is as follows: According to the relationship between the real frequency, the observed frequency and the camera frame rate, in the maximum frequency range [0,frequency max ] to find all possible true frequencies corresponding to the observed frequencies obs1 and obs2, which are recorded as and True frequency F t , observation frequency F o The relationship between the camera frame rate fps is as follows: Among them, MOD(·) is the remainder function; When there exist i and j such that a i and c j Satisfaction relationship|a i -c j |<ε, it is considered that a i and c j equal, ε is the maximum deviation between the ideal value and the actual value of the true frequency; the calculation formula of the vibration frequency F is as follows:
10. A mechanical vibration detection method based on moiré patterns, based on the system according to any one of claims 1 to 9, characterized in that: The method comprises the following steps: 1) using a camera to collect video of a target object with a moiré visual marker attached to its surface at a first frame rate fps1, wherein the video frame contains a moiré pattern image with a low spatial frequency; 2) The region of interest in the video frame sequence is roughly selected according to the ArUco markers, divided into four sub-regions of interest, and the size of the region of interest is fine-tuned according to the robustness of the moiré feature; 3) Perform image enhancement and binarization operations on the video frame sequence; 4) For each sub-region of interest, a lightweight spectrum weighted feature extraction algorithm is used to extract moiré features for each frame of the video, and the vibration curves of the X-axis and Y-axis are calculated using an in-plane vibration model based on the moiré phase; the vibration curve of the Z-axis is calculated using an out-of-plane vibration model based on the moiré spatial frequency; 5) Based on the multi-channel fusion algorithm, the data of the four sub-regions of interest are fused to obtain the fused three-axis vibration curve; 6) estimating the three-axis vibration amplitudes respectively according to the three-axis vibration curves obtained in step 5); 7) Select the second frame rate fps2 for video acquisition based on the dual frame rate selection algorithm, and repeat steps 2) to 5); 8) Estimate vibration frequency based on dual frame rate frequency reconstruction method.
Citation Information
Patent Citations
Method and a system for detecting vibrations in an industrial vision
CN109313070A
Variable density stripe-based three-dimensional vibration vision measuring system and method
CN109341548A
Motion mechanism linearity detecting system and detecting method thereof
CN110487219A
Pose estimation system and method based on moire
CN115457134A
Moire-based ranging system and method
CN115457137A
Cited By
Camera anti-shake method, electronic equipment and computer readable storage medium
CN119383470A
Physiological signal detection method and device based on mole effect, terminal equipment and storage medium
CN120163817A
Physiological signal detection method, device, terminal equipment and storage medium based on Moore effect
CN120163817B
