A depth perception based adaptive out-of-plane vibration measurement system and method
By using a depth-sensing-based adaptive out-of-plane vibration measurement system, which utilizes a binocular stereo vision system to acquire depth image sequences, the problem of traditional methods being unable to capture out-of-plane vibration information is solved, enabling non-contact, rapid, and accurate vibration measurement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN ENG UNIV
- Filing Date
- 2023-10-27
- Publication Date
- 2026-05-19
AI Technical Summary
Traditional vibration measurement methods are difficult to effectively capture out-of-plane vibration information, especially under complex backgrounds, lighting changes, and noise interference, the measurement results are unstable.
An adaptive out-of-plane vibration measurement system based on depth perception is adopted. It uses a binocular stereo vision system to acquire depth image sequences and achieves non-contact out-of-plane vibration measurement through pixel filtering, difference calculation, region determination and frequency calculation.
It can quickly and accurately capture out-of-plane vibration information of objects without contact, reducing the impact of complex backgrounds and noise, and improving the stability and applicability of measurement results.
Smart Images
Figure CN117522944B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an out-of-plane vibration measurement system and method, belonging to the field of out-of-plane vibration measurement technology. Background Technology
[0002] Traditional vibration measurement methods typically require direct contact between the object and the sensor, which leads to measurement results being affected by factors such as friction, device stiffness, and sensor mass. With technological advancements, non-contact methods such as laser Doppler vibration measurement, acoustic vibration measurement, and video vibration measurement have been proposed and applied to the field of vibration measurement, effectively overcoming these shortcomings. However, the slow speed and high cost of laser Doppler vibration measurement, and the limited application scenarios of acoustic vibration measurement, restrict the use and development of these technologies.
[0003] Video vibration measurement utilizes camera equipment to record the motion of vibrating objects and extract vibration information, effectively addressing many drawbacks of traditional vibration measurement. Existing techniques typically process and analyze image sequences of vibrating objects, thus possessing good applicability and full-field measurement capabilities. Currently, the mainstream video vibration measurement methods mainly include image processing-based methods and optical flow-based methods. One type of image processing-based video vibration measurement method typically estimates vibration based on grayscale changes or spatial phase changes on the object's surface. For example, CN111784647, an invention entitled "A Method for Testing Structural Modal Vibration Amplification via Video," uses spatial filtering and amplification of the spatial phase changes in a video sequence to obtain information such as modal parameters and natural frequencies. The other type of optical flow-based video vibration measurement method infers vibration information based on changes in light intensity between object pixels. For example, CN114187330, an invention entitled "A Method for Analyzing Working Modal Characteristics of Structural Micro-amplitude Vibration Based on Optical Flow," uses analysis of displacement changes at structural contour points obtained by optical flow to acquire the modal vibration response. However, these commonly used techniques still have some limitations, such as difficulty in effectively capturing out-of-plane vibration information of objects, which limits the application scenarios; and they are easily interfered with in scenarios with complex backgrounds, changes in lighting, and significant noise, leading to instability in measurement results.
[0004] Therefore, there is an urgent need to propose an adaptive out-of-plane vibration measurement system and method based on depth perception to solve the above-mentioned technical problems. Summary of the Invention
[0005] The purpose of this invention is to address the problem that traditional vibration measurement algorithms struggle to capture out-of-plane vibration information, and to provide a depth-sensing-based adaptive out-of-plane vibration measurement system and method. A brief overview of the invention is provided below to offer a basic understanding of certain aspects of it. It should be understood that this overview is not an exhaustive summary of the invention. It is not intended to identify key or essential parts of the invention, nor is it intended to limit the scope of the invention.
[0006] The technical solution of this invention:
[0007] An adaptive out-of-plane vibration measurement system based on depth perception includes an image acquisition module, a pixel filtering module, a difference calculation module, a region determination module, a vibration localization module, a displacement extraction module, and a frequency calculation module. The image acquisition module uses a binocular stereo vision system to calculate and acquire the out-of-plane vibration process of the structure under test at a frame rate that satisfies the Nyquist sampling rate, thereby obtaining a set of depth image sequences showing continuous vibration changes.
[0008] Pixel filtering module: Based on the effective working distance range of the system, the acquired depth image sequence is filtered pixel by pixel, and pixels with depth values within the working distance range are retained to obtain the intersection of the set of effective pixel coordinates;
[0009] Difference Calculation Module: Calculates the difference between all depth images starting from the second frame in the depth image sequence and the first frame depth image, then takes the absolute value of the difference and superimposes them into a single depth difference image;
[0010] Region determination module: Performs logical index operation with the intersection of the set of valid pixel coordinates and the depth difference image to remove invalid pixels in the depth difference image, thereby obtaining a depth difference image of a valid region;
[0011] Vibration localization module: Based on the proportional threshold, it filters out significant non-zero pixels in the effective area depth difference image, and then finds its largest connected region to determine the out-of-plane vibration region;
[0012] Displacement extraction module: Takes the average depth value of the out-of-plane vibration region in the original depth image sequence, and extracts the average depth value sequence of the out-of-plane vibration region as the displacement signal of the out-of-plane vibration of the measured result;
[0013] Frequency calculation module: Performs a fast Fourier transform on the depth value sequence to obtain the out-of-plane vibration frequency of the measured structure.
[0014] An adaptive out-of-plane vibration measurement method based on depth sensing includes the following steps:
[0015] Step 1: Calculate and acquire a set of depth image sequences showing continuous vibration during the out-of-plane vibration of the structure under test using a binocular stereo vision system, while satisfying the Nyquist sampling rate.
[0016] Step 2: Filter the acquired depth image sequence pixel by pixel according to the effective working distance range of the system, and retain the pixels whose depth values are within the working distance range to obtain the intersection of the set of effective pixel coordinates;
[0017] Step 3: Calculate the difference between all depth images starting from the second frame in the depth image sequence and the first frame depth image, then take the absolute value of the difference and superimpose them into a single depth difference image;
[0018] Step 4: Perform a logical index operation between the intersection of the set of valid pixel coordinates and the depth difference image to remove invalid pixels in the depth difference image, thereby obtaining a depth difference image of the valid region.
[0019] Step 5: Based on the ratio threshold, select significant non-zero pixels in the effective region depth difference image, then find its largest connected region and determine it as an out-of-plane vibration region;
[0020] Step 6: Take the average depth value of the out-of-plane vibration region in the original depth image sequence, and extract the average depth value sequence of the out-of-plane vibration region as the displacement signal of the out-of-plane vibration of the measured result.
[0021] Step 7: Perform a fast Fourier transform on the average depth value sequence to obtain the out-of-plane vibration frequency of the structure under test.
[0022] Preferred method: In step 1, the left and right imagers of a binocular stereo vision system capture images of the same scene from different perspectives, and calculate the depth value of each pixel in the image, including the following steps:
[0023] Step 1.1: Calibrate the left and right cameras of the binocular stereo vision system to obtain the intrinsic and extrinsic parameters and the baseline length B. Then, correct the images captured by the left and right cameras according to the calibration results so that they are located on the same plane and parallel to each other.
[0024] Step 1.2: Perform feature pixel matching on the corrected image and calculate the disparity value d. Then, calculate the distance Z from the object to the imaging plane based on the triangulation principle to obtain the depth image sequence, i.e.
[0025]
[0026] Where f is the focal length, and in binocular stereo vision systems, left and right imaging devices with the same focal length are usually used.
[0027] Preferably, in step 2, the depth image sequence is processed to obtain the intersection of the sets of effective pixels, including the following steps:
[0028] Step 2.1: Perform element-by-element determination on the first frame depth image and store the coordinates of pixels whose depth values are within the effective working distance range of the system in a collection container;
[0029] Step 2.2: Traverse all depth images from the second frame to the last frame, repeating the above determination process. The number of repetitions is equal to the number of frames in the depth image sequence. Continuously update the coordinate elements in the container to obtain the intersection C of the set of effective pixels for the depth image sequence, i.e.
[0030]
[0031] in, D represents finding the intersection of all sets. i d represents the depth image in sequence i. min and d max These are the system's minimum effective working distance and maximum effective working distance, respectively.
[0032] Preferably, in step 3, the depth image sequence is processed to obtain a depth difference image, including the following steps:
[0033] Step 3.1: Calculate the absolute difference between the second frame and the first frame of the depth image sequence;
[0034] Step 3.2: For all depth images from the second frame to the last frame, repeat the absolute difference calculation process described above. The number of repetitions is one less than the number of frames in the depth image sequence. After superimposing all the absolute difference images, divide by the number of samples to obtain an average depth difference image (DOD).
[0035]
[0036] Where ||·||2 represents taking the absolute value, D1 represents the depth image in sequence with the order 1, and N represents the number of frames contained in the depth image sequence.
[0037] Preferably, in step 4, an element-wise logical masking operation is performed on the intersection of the effective pixel set and the depth difference image to obtain a depth difference image of the effective region, including the following steps:
[0038] Step 4.1: Obtain the logical mask image T by using the intersection of the effective pixel sets;
[0039] Step 4.2: Perform element-wise multiplication of the logical mask image and the depth difference image (DOD) to remove invalid pixels from the depth difference image, resulting in a depth difference image R representing the effective region.
[0040]
[0041] Here, ⊙ represents element-wise multiplication.
[0042] Preferably, in step 6, during the calculation of frequency domain information using the obtained depth value sequence, the DC component needs to be removed from the depth value sequence, including the following steps:
[0043] Step 6.1: Sum the entire depth value sequence and divide by the number of samples in the sequence to obtain the average value;
[0044] Step 6.2: Then, subtract the average value from the original sequence to obtain the depth value sequence after removing the DC component, i.e.
[0045]
[0046] The present invention has the following beneficial effects:
[0047] 1. The method described in this invention uses a binocular stereo vision system as a data acquisition device, which can effectively capture the depth change information of the object's out-of-plane vibration without contact, and then calculate the vibration amplitude and frequency.
[0048] 2. This invention does not require unnecessary parameter adjustments and has a small overall computational load, thus enabling rapid and accurate location of the vibration area within the field of view, achieving adaptive ground vibration measurement. Attached Figure Description
[0049] Figure 1 This is a schematic diagram of an adaptive out-of-plane vibration measurement system based on depth sensing.
[0050] Figure 2 This is a diagram showing the processing results of the adaptive out-of-plane vibration measurement method on a cantilever beam;
[0051] Figure 3 These are the time-domain and frequency-domain measurement results of the adaptive out-of-plane vibration measurement method. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention is described below with reference to specific embodiments shown in the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.
[0053] Specific implementation method one: Combining Figure 1 This embodiment describes an adaptive out-of-plane vibration measurement system based on depth perception, comprising an image acquisition module, a pixel filtering module, a difference calculation module, a region determination module, a vibration localization module, a displacement extraction module, and a frequency calculation module. The image acquisition module, a hardware component of the system, consists of a binocular stereo vision system and a computer connected via a USB 3.2 cable. Camera control software on the computer sends acquisition commands to the binocular stereo vision system, records and saves the captured raw depth image sequence, and then imports it into MATLAB software for processing. The pixel filtering module, difference calculation module, region determination module, vibration localization module, displacement extraction module, and frequency calculation module are sequentially called as different functional modules within the MATLAB software. This invention effectively captures out-of-plane vibration information of structures, exhibits strong adaptability, and is not easily affected by complex backgrounds, lighting changes, noise, etc., reducing costs while ensuring stable measurement results.
[0054] Image acquisition module: Utilizes a binocular stereo vision system to calculate and acquire the out-of-plane vibration process of the structure under test at a frame rate that satisfies the Nyquist sampling rate, obtaining a set of depth image sequences showing continuous vibration changes;
[0055] Pixel filtering module: Based on the effective working distance range of the system, the acquired depth image sequence is filtered pixel by pixel, and pixels with depth values within the working distance range are retained to obtain the intersection of the set of effective pixel coordinates;
[0056] Difference Calculation Module: Calculates the difference between all depth images starting from the second frame in the depth image sequence and the first frame depth image, then takes the absolute value of the difference and superimposes them into a single depth difference image;
[0057] Region determination module: Performs logical index operation with the intersection of the set of valid pixel coordinates and the depth difference image to remove invalid pixels in the depth difference image, thereby obtaining a depth difference image of a valid region;
[0058] Vibration localization module: Based on the proportional threshold, it filters out significant non-zero pixels in the effective area depth difference image, and then finds its largest connected region to determine the out-of-plane vibration region;
[0059] Displacement extraction module: Takes the average depth value of the out-of-plane vibration region in the original depth image sequence, and extracts the average depth value sequence of the out-of-plane vibration region as the displacement signal of the out-of-plane vibration of the measured result;
[0060] Frequency calculation module: Performs a fast Fourier transform on the depth value sequence to obtain the out-of-plane vibration frequency of the measured structure.
[0061] Specific Implementation Method Two: Combining Figure 1-3 This embodiment describes a depth-sensing-based adaptive out-of-plane vibration measurement method, employing the aforementioned depth-sensing-based adaptive out-of-plane vibration measurement system, and includes the following steps:
[0062] Step 1: Calculate and acquire the out-of-plane vibration process of the structure under test using a binocular stereo vision system at a frame rate that satisfies the Nyquist sampling rate, and obtain a set of depth image sequences showing continuous vibration changes. Compared with traditional phase-based vibration measurement methods, the method described in this invention uses a binocular stereo vision system as an acquisition device, which can effectively capture the out-of-plane depth information of the object without contact, and then calculate the vibration amplitude and frequency.
[0063] Step 2: Filter the acquired depth image sequence pixel by pixel according to the effective working distance range of the system, and retain the pixels whose depth values are within the working distance range to obtain the intersection of the set of effective pixel coordinates;
[0064] Step 3: Calculate the difference between all depth images starting from the second frame in the depth image sequence and the first frame depth image, then take the absolute value of the difference and superimpose them into a single depth difference image;
[0065] Step 4: Perform a logical index operation between the intersection of the set of valid pixel coordinates and the depth difference image to remove invalid pixels in the depth difference image, thereby obtaining a depth difference image of the valid region.
[0066] Step 5: Based on the ratio threshold, select significant non-zero pixels in the effective region depth difference image, and then find its largest connected region to determine the out-of-plane vibration region;
[0067] Step 6: Take the average depth value of the out-of-plane vibration region in the original depth image sequence, and extract the average depth value sequence of the out-of-plane vibration region as the displacement signal of the out-of-plane vibration of the measured result.
[0068] Step 7: Perform a Fast Fourier Transform on the average depth value sequence to obtain the out-of-plane vibration frequency of the structure under test;
[0069] This invention requires no extra parameter adjustments and has a small overall computational load, thus enabling rapid and accurate location of the vibration area within the field of view, achieving adaptive ground vibration measurement.
[0070] Specific implementation method three: Combining Figure 1-3This embodiment describes an adaptive out-of-plane vibration measurement method based on depth perception. Step 1 involves capturing images of the same scene from different viewpoints using the left and right imagers of a binocular stereo vision system, and calculating the depth value of each pixel in the images. This includes the following steps:
[0071] Step 1.1: Calibrate the left and right cameras of the binocular stereo vision system to obtain the intrinsic and extrinsic parameters and the baseline length B. Then, correct the images captured by the left and right cameras according to the calibration results so that they are located on the same plane and parallel to each other.
[0072] Step 1.2: Perform feature pixel matching on the corrected image and calculate the disparity value d. Then, calculate the distance Z from the object to the imaging plane based on the triangulation principle to obtain the depth image sequence, i.e.
[0073]
[0074] Where f is the focal length, and in binocular stereo vision systems, left and right imaging devices with the same focal length are usually used.
[0075] Specific implementation method four: Combination Figure 1-3 This embodiment describes an adaptive out-of-plane vibration measurement method based on depth perception. Step 2 involves processing the depth image sequence to obtain the intersection of the effective pixel sets, including the following steps:
[0076] Step 2.1: Perform element-by-element determination on the first frame depth image, and store the pixel coordinates whose depth values are within the effective working distance range of the system (an adaptive out-of-plane vibration measurement system based on depth perception) in a collection container;
[0077] Step 2.2: Traverse all depth images from the second frame to the last frame, repeating the above determination process (Step 2.1), the number of repetitions being equal to the number of frames in the depth image sequence, continuously updating the coordinate elements in the container, to obtain the intersection C of the set of effective pixels for the depth image sequence, i.e.
[0078]
[0079] in, D represents finding the intersection of all sets. i d represents the depth image in sequence i. min and d max These are the minimum and maximum effective working distances of the system, respectively, and (x, y) are the pixel coordinates of the depth image, representing a pixel.
[0080] Specific Implementation Method Five: Combining Figure 1-3This embodiment describes an adaptive out-of-plane vibration measurement method based on depth sensing. Step 3 involves processing the depth image sequence to obtain a depth difference image, including the following steps:
[0081] Step 3.1: Calculate the absolute difference between the second frame and the first frame of the depth image sequence;
[0082] Step 3.2: For all depth images from the second frame onwards to the last frame, repeat the absolute difference calculation process (Step 3.1) for one more time than the number of frames in the depth image sequence. Superimpose all absolute difference images and divide by the number of samples to obtain an average depth difference image (DOD).
[0083]
[0084] Where ||·||2 represents taking the absolute value, D1 represents the depth image in sequence with the order 1, and N represents the number of frames contained in the depth image sequence.
[0085] Specific Implementation Method Six: Combination Figure 1-3 This embodiment describes an adaptive out-of-plane vibration measurement method based on depth perception. In step 4, an element-wise logical masking operation is performed on the intersection of the effective pixel set and the depth difference image to obtain a depth difference image of the effective region. This includes the following steps:
[0086] Step 4.1: Obtain the logical mask image T by using the intersection of the effective pixel sets;
[0087] Step 4.2: Perform element-wise multiplication of the logical mask image and the depth difference image (DOD) to remove invalid pixels from the depth difference image, resulting in a depth difference image R representing the effective region.
[0088]
[0089] Here, ⊙ represents element-wise multiplication.
[0090] Specific implementation method seven: Combining Figure 1-3 This embodiment describes an adaptive out-of-plane vibration measurement method based on depth sensing. In step 6, during the calculation of frequency domain information using the obtained depth value sequence, the DC component of the depth value sequence needs to be removed. This includes the following steps:
[0091] Step 6.1: Sum the entire depth value sequence and divide by the number of samples in the sequence to obtain the average value;
[0092] Step 6.2: Then, subtract this average value from the original depth value sequence to obtain the depth value sequence after removing the DC component, in order to avoid zero-frequency components in the frequency domain results.
[0093]
[0094] Where x(n) is the sequence of average depth values of the out-of-plane vibration region.
[0095] Example 1:
[0096] To better describe the method described in this application, the following embodiments are used to illustrate the complete process of a depth-sensing-based adaptive out-of-plane vibration measurement method in practical applications, including the following steps:
[0097] Step 1: Prepare a cantilever beam with a length of 40cm, apply external excitation to trigger vibration;
[0098] Step 1.1: Use camera control software to pre-calibrate the left and right cameras of the binocular stereo vision system, then set the acquisition resolution to 848*480 pixels and the frame rate to 90Hz, and perform time-series acquisition and correction of the vibration scene with the field of view facing the cantilever beam.
[0099] Step 1.2: Perform feature pixel matching on the corrected image and calculate the disparity value d. Then, calculate the distance Z from the object to the imaging plane based on the triangulation principle to obtain the depth image sequence, i.e.
[0100]
[0101] Where f is the focal length, and in a binocular stereo vision system, left and right imaging devices with the same focal length are usually used.
[0102] Step 2:
[0103] Step 2.1: Perform element-by-element determination on the aforementioned depth image sequence;
[0104] Step 2.2: Obtain the intersection C of the set of effective pixels for the depth image sequence:
[0105]
[0106] in D represents finding the intersection of all sets. i Let represent the i-th depth image in the depth video sequence. The minimum effective working distance and the maximum effective working distance of the binocular stereo vision system are set to 300mm and 1500mm, respectively.
[0107] Step 3:
[0108] Step 3.1: Calculate the absolute difference between all depth images from the second to the last frame in the depth image sequence and the first frame depth image;
[0109] Step 3.2: Overlay all absolute difference images to obtain a depth difference image (DOD).
[0110]
[0111] Where ||·||2 represents taking the absolute value, and N is the number of frames in the depth sequence;
[0112] Step 4:
[0113] Step 4.1: Obtain the logical mask image T by using the intersection of the effective pixel sets;
[0114] Step 4.2: Perform element-wise multiplication with the depth difference image DOD to remove invalid pixels from the depth difference image, obtaining a depth difference image R of the effective region:
[0115]
[0116] Where ⊙ represents element-wise multiplication;
[0117] Step 5: As Figure 2 The process shown is based on the eight-neighbor connection operation, which performs connected component analysis on the effective region depth difference image, extracts the largest connected region in the image, and determines it as the region that continuously generates vibration in the acquisition sequence.
[0118] Step 6:
[0119] Step 6.1: Take the mean of the depth values in the out-of-plane vibration region of the original depth image sequence, extract the average depth value sequence of the out-of-plane vibration region as the displacement signal of the out-of-plane vibration of the measured result, and use the mean(·) operator to obtain the mean of the displacement signal.
[0120] Step 6.2: Subtract this mean from the original sequence to obtain the depth value sequence after removing the DC component, thus avoiding the presence of zero-frequency components.
[0121]
[0122] Step 7: As Figure 3 The results shown are obtained by performing frequency domain calculations on the depth value sequence using Fourier transform;
[0123] This invention does not require pre-adjustment of parameters according to the scenario, has a fast operating speed, and is highly practical, enabling accurate and efficient adaptive non-contact out-of-plane vibration measurement.
[0124] It should be noted that in the above embodiments, as long as the technical solutions are not contradictory, they can be permuted and combined. Those skilled in the art can exhaust all possibilities based on the mathematical knowledge of permutation and combination. Therefore, the present invention will not describe the technical solutions after permutation and combination one by one, but it should be understood that the technical solutions after permutation and combination have been disclosed by the present invention.
[0125] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. An adaptive out-of-plane vibration measurement system based on depth sensing, characterized in that: It includes an image acquisition module, a pixel filtering module, a difference calculation module, a region determination module, a vibration positioning module, a displacement extraction module, and a frequency calculation module. The image acquisition module uses a binocular stereo vision system to calculate and acquire the out-of-plane vibration process of the structure under test at a frame rate that meets the Nyquist sampling rate, and obtains a set of depth image sequences with continuously changing vibration. Pixel filtering module: Based on the effective working distance range of the system, the acquired depth image sequence is filtered pixel by pixel, and pixels with depth values within the working distance range are retained to obtain the intersection of the set of effective pixel coordinates; Difference Calculation Module: Calculates the difference between all depth images starting from the second frame in the depth image sequence and the first frame depth image, then takes the absolute value of the difference and superimposes them into a single depth difference image; Region determination module: Performs logical index operation with the intersection of the set of valid pixel coordinates and the depth difference image to remove invalid pixels in the depth difference image, thereby obtaining a depth difference image of a valid region; Vibration localization module: Based on the proportional threshold, it filters out significant non-zero pixels in the effective area depth difference image, then finds its largest connected region and determines it as an out-of-plane vibration region; Displacement extraction module: Takes the average depth value of the out-of-plane vibration region in the original depth image sequence, and extracts the average depth value sequence of the out-of-plane vibration region as the displacement signal of the out-of-plane vibration of the measured result; Frequency calculation module: Performs a fast Fourier transform on the depth value sequence to obtain the out-of-plane vibration frequency of the measured structure.
2. An adaptive out-of-plane vibration measurement method based on depth sensing, characterized in that: The adaptive out-of-plane vibration measurement system based on depth sensing as described in claim 1 includes the following steps: Step 1: Use a binocular stereo vision system to calculate and acquire the out-of-plane vibration process of the structure under test at a frame rate that meets the Nyquist sampling rate, and obtain a set of depth image sequences with continuously changing vibrations. Step 2: Filter the acquired depth image sequence pixel by pixel according to the effective working distance range of the system, and retain the pixels whose depth values are within the working distance range to obtain the intersection of the set of effective pixel coordinates; Step 3: Calculate the difference between all depth images starting from the second frame in the depth image sequence and the first frame depth image, then take the absolute value of the difference and superimpose them into a single depth difference image; Step 4: Perform a logical index operation between the intersection of the set of valid pixel coordinates and the depth difference image to remove invalid pixels in the depth difference image, thereby obtaining a depth difference image of the valid region. Step 5: Based on the ratio threshold, select significant non-zero pixels in the effective region depth difference image, and then find its largest connected region to determine the out-of-plane vibration region; Step 6: Take the average depth value of the out-of-plane vibration region in the original depth image sequence, and extract the average depth value sequence of the out-of-plane vibration region as the displacement signal of the out-of-plane vibration of the measured result. Step 7: Perform a Fast Fourier Transform on the depth value sequence to obtain the out-of-plane vibration frequency of the structure under test.
3. The adaptive out-of-plane vibration measurement method based on depth sensing according to claim 2, characterized in that: In step 1, the left and right imagers of a binocular stereo vision system capture images of the same scene from different perspectives, and calculate the depth value of each pixel in the image, including the following steps: Step 1.1: Calibrate the left and right cameras of the binocular stereo vision system to obtain the intrinsic and extrinsic parameters and the baseline length B. Then, correct the images captured by the left and right cameras according to the calibration results so that they are located on the same plane and parallel to each other. Step 1.2: Perform feature pixel matching on the corrected image and calculate the disparity value d. Then, calculate the distance Z from the object to the imaging plane based on the triangulation principle to obtain the depth image sequence, i.e. Where f is the focal length, and in binocular stereo vision systems, left and right imaging devices with the same focal length are usually used.
4. The adaptive out-of-plane vibration measurement method based on depth sensing according to claim 3, characterized in that: Step 2 involves processing the depth image sequence to obtain the intersection of the effective pixel sets, including the following steps: Step 2.1: Perform element-by-element determination on the first frame depth image and store the coordinates of pixels whose depth values are within the effective working distance range of the system in a collection container; Step 2.2: Traverse all depth images from the second frame to the last frame, repeat the above determination process, update the coordinate elements in the container, and obtain the intersection C of the set of valid pixels for the depth image sequence, i.e. in, D represents finding the intersection of all sets. i d represents the depth image in sequence i. min and d max These are the minimum and maximum effective working distances of the system, respectively, and (x, y) are the pixel coordinates of the depth image, representing a pixel.
5. The adaptive out-of-plane vibration measurement method based on depth sensing according to claim 4, characterized in that: Step 3 involves processing the depth image sequence to obtain a depth difference image, including the following steps: Step 3.1: Calculate the absolute difference between the second frame and the first frame of the depth image sequence; Step 3.2: For all depth images from the second frame onwards to the last frame, repeat the absolute difference calculation process described above. Superimpose all absolute difference images and divide by the number of samples to obtain an average depth difference image (DOD). Where ||·||2 represents taking the absolute value, and N represents the number of frames contained in the depth image sequence.
6. The adaptive out-of-plane vibration measurement method based on depth sensing according to claim 5, characterized in that: In step 4, an element-wise logical masking operation is performed on the intersection of the effective pixel set and the depth difference image to obtain a depth difference image of the effective region, including the following steps: Step 4.1: Obtain the logical mask image T by using the intersection of the effective pixel sets; Step 4.2: Perform element-wise multiplication of the logical mask image and the depth difference image (DOD) to remove invalid pixels from the depth difference image, resulting in a depth difference image R representing the effective region. R=T⊙DOD, Here, ⊙ represents element-wise multiplication.
7. The adaptive out-of-plane vibration measurement method based on depth sensing according to claim 6, characterized in that: In step 6, during the calculation of frequency domain information using the obtained depth value sequence, the DC component needs to be removed from the depth value sequence, including the following steps: Step 6.1: Sum the entire depth value sequence and divide by the number of samples in the sequence to obtain the average value; Step 6.2: Then, subtract the average value from the original sequence to obtain the depth value sequence after removing the DC component, i.e. x(n) is the depth value sequence.