Device and method for real-time reconstruction of vibration signals

Abstract

The invention discloses a device and a method for real-time reconstruction of a vibration signal. According to the device, a pulse camera is used for collecting image data, and high-speed pulse array data are transmitted to a signal processing module through a data interface module. And the signal processing module processes the data, generates a low-speed traditional image stream to be displayed by the display, and recovers a sound vibration signal of the measured object. Compared with the prior art, through high-speed data acquisition of the pulse camera and efficient processing of the signal processing module, real-time recovery of sound vibration signals is realized, and a new solution is provided for a visual vibration measurement technology.

Description

Technical Field

[0001] The present invention relates to a device for real-time reconstruction of vibration signals, and also relates to a method for real-time reconstruction of vibration signals, belonging to the technical field of visual vibration measurement. Background Art

[0002] Visual Vibrometry is a non-contact vibration measurement technology that obtains the vibration information of an object through a video. With the continuous progress of visual sensing and image processing technologies, visual vibrometry, as a non-contact measurement means, has received extensive research and attention and has been applied in many engineering fields such as machinery, vehicles, buildings, and aerospace.

[0003] In order to more accurately characterize the motion trajectory of a vibrating object, technicians generally use existing high-speed cameras. They achieve high-speed sampling of signals through high frame rates to meet the measurement requirements of high-frequency dynamic responses. For example, some research uses a 20,000-frame high-speed camera to record the static video of an object from a distance, and then uses visual vibrometry technology to recover the acoustic vibration signal from the silent video. However, existing high-speed cameras use a timed exposure method to obtain videos in the form of image sequences. This method cannot effectively express the process of an extremely high-speed photon stream, resulting in the loss of vibration information. To solve the contradiction between speed and dynamic range caused by this exposure strategy, a research team has proposed a pulse array vision technology for all-time imaging. As Figure 1 shown, the core idea of this technology is to use 1-bit quantized high-speed signals to describe the light intensity signals corresponding to each pixel point, thus breaking through the concept of frames in traditional images. For each pixel point, an all-time recording and asynchronous transmission scheme is adopted, breaking through the limitation of the exposure time window. In addition, since each pixel uses 1-bit quantization, the output circuit is very simple. Compared with traditional image sensors, at the same equivalent bit width, its sampling rate is higher and the implementation cost is lower.

[0004] In a Chinese patent application with the publication number CN115022557A, a method and system for improving the visual vibration measurement frequency range are disclosed. The core of this technical solution lies in presetting the division of an image sensor to obtain multiple regions. Then, the working timing parameters of the image sensor are configured. These parameters cover the working time periods of each region and ensure that these time periods do not overlap. According to these working timing parameters, each region of the image sensor can start and stop the working state in sequence, thereby realizing the image acquisition of different local positions of the object to be measured at multiple different moments. Finally, by combining the images of the same local position of the object to be measured at multiple different moments and the time correlation of the vibration information, the vibration parameters of this local position are obtained. At the same time, by using the spatial correlation of the vibration information of different local positions, the global vibration parameters of the object to be measured are analyzed and determined. However, this technical solution mainly focuses on improving the frequency range of visual vibration measurement and does not further optimize its real-time performance. Summary of the Invention

[0005] The primary technical problem to be solved by the present invention is to provide a device for real-time reconstruction of vibration signals.

[0006] Another technical problem to be solved by the present invention is to provide a method for real-time reconstruction of vibration signals.

[0007] To achieve the above technical objectives, the present invention adopts the following technical solutions:

[0008] According to the first aspect of the embodiments of the present invention, a device for real-time reconstruction of vibration signals is provided, including:

[0009] A pulsed camera for collecting image data and sending the collected image data to a data interface module;

[0010] A data interface module for receiving the high-speed pulse array data sent by the pulsed camera and sending the high-speed pulse array data to a signal processing module;

[0011] A display for receiving and displaying the low-speed traditional image stream sent by the signal processing module;

[0012] A signal processing module for receiving the high-speed pulse array data sent by the data interface module, processing the data, generating a low-speed traditional image stream and sending it to the display, and restoring the acoustic vibration signal of the object to be measured and sending it to a speaker;

[0013] A speaker for receiving and playing the voice signal of the object to be measured sent by the signal processing module.

[0014] Preferably, the signal processing module includes:

[0015] A data acquisition unit, configured to receive the high-speed pulse array data sent by the data interface module and send the data to the data interface unit;

[0016] A data interface unit, configured to receive the high-speed pulse array data sent by the data acquisition unit, send the data to the pixel block screening unit, and send the selected sub-array stream through corner point search to the acoustic vibration signal recovery unit, and receive the position information of the effective vibration pixels sent by the pixel block screening unit;

[0017] A pixel block screening unit, configured to convert the high-speed pulse array data into low-speed pulse array data by downsampling and convert the low-speed pulse array data into a low-speed traditional image stream and send it to the display;

[0018] An acoustic vibration signal recovery unit, configured to collect multiple sub-array streams, recover the corresponding acoustic vibration signals, and send the recovered acoustic vibration signals to the signal post-processing unit;

[0019] A signal post-processing unit, configured to synthesize multiple acoustic vibration signals into one voice signal and send the synthesized voice signal to the speaker.

[0020] According to the second aspect of the embodiments of the present invention, there is provided a method for real-time reconstruction of vibration signals, including the following steps:

[0021] S1: Power on and start up;

[0022] S2: Convert the high-speed pulse array data captured by the pulse camera into a low-speed traditional image stream, and adjust the parameters so that the target area is clearly imaged on the display;

[0023] S3: Search for corner points and calibrate the center positions of the corner points;

[0024] S4: Determine whether the number of corner points meets the requirements; if the number of corner points is greater than or equal to a preset threshold, go to step S5; otherwise, return to step S2 to re-search for corner points by adjusting the target area and parameters;

[0025] S5: Generate pulse sub-arrays and organize them into a data stream;

[0026] S6: Reconstruct the acoustic vibration signal corresponding to the vibration of the object under test by using the information extracted from the data stream;

[0027] S7: Perform multi-channel signal merging and noise reduction processing to integrate and optimize the acoustic vibration signals recovered from multiple pulse sub-arrays;

[0028] S8: Judge the signal-to-noise ratio of the acoustic vibration signal by outputting the signal through the speaker; if the signal-to-noise ratio meets the preset value, keep the voice output; otherwise, return to step S2.

[0029] Compared with the prior art, the present invention uses a pulsed camera to replace the existing high-speed camera, achieving the acquisition of spatial optical signals and more realistically recording acoustic vibration signals using pixels. This pulsed camera enables each pixel to work independently, converting the received photon information into pulsed signals, thereby capturing high-speed visual phenomena at an extremely high sampling frequency and breaking through the frame rate limitation of traditional cameras. In addition, the present invention proposes an effective pixel block screening mechanism, which reduces the requirement for the data interface speed between the camera and the processor and the bandwidth requirement for data transmission through specific data processing methods, making data processing more efficient. At the same time, the present invention uses multi-channel signal processing technology instead of traditional image processing technology to process the data captured from the pulsed camera, and precisely recovers the acoustic vibration signal corresponding to the vibration of the object under test through a series of steps. Multi-channel signal merging and noise reduction processing further improve the clarity and accuracy of the signal, achieving real-time recovery of the acoustic vibration signal and improving the real-time performance of vibration measurement. Description of the Drawings

[0030] Figure 1 It is a logical schematic diagram of pulsed array vision technology in the prior art;

[0031] Figure 2 It is a schematic diagram of an apparatus for real-time reconstruction of vibration signals provided by an embodiment of the present invention;

[0032] Figure 3 It is a schematic diagram of the structure of a signal processing module in an embodiment of the present invention;

[0033] Figure 4 It is a schematic diagram of the time-domain decomposition of pulsed camera data in an embodiment of the present invention;

[0034] Figure 5 It is a flowchart of a method for real-time reconstruction of vibration signals provided by an embodiment of the present invention;

[0035] Figure 6 It is a logic block diagram of the signal processing process in an embodiment of the present invention;

[0036] Figure 7 It is a schematic diagram of the speech recovery process of a pulsed subarray in an embodiment of the present invention;

[0037] Figure 8 It is a schematic diagram of the processing process of a pulsed sequence in an embodiment of the present invention;

[0038] Figure 9 It is a schematic diagram of the recovery process of pixel-level acoustic vibration signals in an embodiment of the present invention. Detailed Embodiments

[0039] The technical content of the present invention will be described in detail below in conjunction with the drawings and specific embodiments.

[0040] First Embodiment

[0041] As Figure 2 shown, the first embodiment of the present invention provides a device for real-time reconstruction of vibration signals. The device consists of a pulsed camera 1, a data interface module 2 based on a field-programmable gate array (FPGA), a display 3, a signal processing module 4, and a speaker 5. Among them, the pulsed camera 1 is responsible for collecting image data and sending it to the data interface module 2. The data interface module 2 receives these high-speed pulse array data and transmits it to the signal processing module 4. The signal processing module 4 further processes this data, sends the low-speed traditional image stream to the display 3, and at the same time sends the voice signal of the object to be measured to the speaker 5 to achieve real-time reconstruction of vibration signals.

[0042] The pulsed camera 1 is one of the core components of the present invention. It mainly consists of a lens and a sensor and is specialized for high-speed data acquisition. Compared with traditional high-speed cameras, the pulsed camera 1 adopts a subversive design. Each pixel works independently, converts the received photon information into a pulse signal, and outputs a pulse array of "0" or "1". This design enables the pulsed camera 1 to capture high-speed visual phenomena at a sampling frequency of up to 200KHz or even higher, which is difficult to achieve with traditional high-speed cameras.

[0043] The data interface module 2 plays the role of data transmission and caching in the present invention. It consists of an FPGA and a storage chip, is connected to the pulsed camera 1 through a parallel interface, ensuring fast transmission of high-speed data. At the same time, the data interface module 2 is connected to the signal processing module 4 through a gigabit network port, further optimizing the data transmission efficiency.

[0044] The signal processing module 4 is the data processing center of the device. It receives the high-speed pulse array data from the data interface module 2 and processes it. The processed data is converted into a low-speed traditional image stream and sent to the display 3 for the user to observe. At the same time, the signal processing module 4 also sends the voice signal of the object to be measured to the speaker 5, enabling the user to hear the restored acoustic vibration signal.

[0045] In summary, the device for real-time reconstruction of vibration signals provided by the embodiment of the present invention realizes efficient acquisition, processing, and output of vibration signals through the collaborative work of each component.

[0046] In an embodiment of the present invention, the signal processing module 4 consists of multiple cooperating units to achieve real-time reconstruction of vibration signals. These units include a data acquisition unit, a data interface unit, a pixel block screening unit, an acoustic vibration signal restoration unit, and a signal post-processing unit.

[0047] The structure of the signal processing module 4 is as Figure 3As shown in the figure. Among them, the data acquisition unit receives the high-speed pulse array data sent by the data interface module 2 and sends the high-speed pulse array data to the data interface unit. The data interface unit serves as a data flow transfer station. It not only sends the high-speed pulse array data to the pixel block screening unit, but also receives the position information of the effective vibration pixels from the pixel block screening unit, and transfers the selected sub-array stream through corner point search to the acoustic vibration signal recovery unit. The pixel block screening unit is responsible for converting the high-speed pulse array data into low-speed pulse array data through downsampling, further converting it into a low-speed traditional image stream, and finally sending it to the display 3 for users to observe. This process not only reduces the bandwidth requirement for data transmission, but also makes the image data easier to process and display. The acoustic vibration signal recovery unit collects multiple sub-array streams, recovers the corresponding acoustic vibration signals from them, and sends them to the signal post-processing unit for further processing. The signal post-processing unit synthesizes multiple acoustic vibration signals into one voice signal, that is, the voice signal of the measured object, and sends it to the speaker 5, enabling users to hear the recovered acoustic vibration signals. Through this modular design, the signal processing module 4 can efficiently process the high-speed pulse array data and realize the real-time reconstruction of vibration signals.

[0048] The signal processing module 4 plays a core role in the present invention, and its main functions include the following three aspects:

[0049] 1) First of all, the signal processing module 4 uses virtual exposure technology to convert the high-speed pulse array received from the pulse camera 1 into a low-rate traditional video signal. This process enables the high-speed data stream to be presented in a form that is easier to process and display, while retaining sufficient information for subsequent analysis.

[0050] 2) Secondly, the signal processing module 4 is responsible for the rapid detection, tracking, and identification of effective vibration pixels. Through precise algorithms, the signal processing module 4 can identify the pixel points that show significant changes during the vibration process. Based on the position information of these effective vibration pixels, the original pulse array is divided into multiple sub-arrays, and each sub-array contains a group of related vibration pixels, providing a basis for the recovery of acoustic vibration signals.

[0051] 3) Finally, the signal processing module 4 recovers the acoustic vibration signals from each sub-array stream. This step involves analyzing and processing the signals of each sub-array to extract the acoustic vibration signals. Subsequently, these signals are merged and noise-reduced to improve the clarity and accuracy of the signals. The processed acoustic vibration signals are output for further analysis or directly used for sound output.

[0052] Figure 4 This is the time-domain decomposition diagram of the pulse camera data in the embodiment of the present invention. Through the decomposition in the time domain, it is convenient to understand how the pulse camera records photon information over time. In Figure 4Among them, the three dimensions are the X-axis, the Y-axis, and the time axis t, representing two dimensions of space and the time dimension respectively. Each cube represents a pulse plane, which records the arrival of photons at pixel points on the X-axis and Y-axis at a specific time point t. Each pixel point in the pulse plane can record the presence or absence of photons, usually represented by binary 1 or 0. These pulse planes change continuously as time progresses, forming a dynamic sequence of photon arrival information.

[0053] The pulse sequence is composed of consecutive pulse planes, which are unfolded along the time axis t. These sequences provide us with the photon arrival information of the object at different time points and are the basis for analyzing the vibration characteristics of the object. In addition, Figure 4 also shows a pulse sub-array, that is, a subset containing specific pixel blocks extracted from the original pulse array. This pulse sub-array can be used for more detailed vibration signal analysis because it can provide more localized photon arrival information.

[0054] The time axis t plays the role of recording the passage of time. Each pulse plane is recorded at a different time point t, thus forming a sequence of photon arrival information that changes over time. This way of recording time series enables the pulsed camera to capture the process of a high-speed dynamic photon stream, which is difficult for traditional cameras to achieve. The spatial axes X and Y define the positions of pixel points on the pulsed camera sensor. Together with the time axis t, they constitute a three-dimensional space for the pulsed camera to record photon arrival information. Through this three-dimensional space recording method, the pulsed camera can provide a brand-new perspective to observe and analyze the vibration characteristics of an object.

[0055] Figure 4 clearly shows how the pulsed camera completes the acquisition of image data and outputs a pulse array. In this process, each element in the pulse array only contains two states, "1" or "0", and these states represent whether photons are detected at the corresponding pixel positions at a specific time point. For each pixel position, the "1" and "0" states at different time points form a pulse sequence. At a specific moment, the states of all pixels together constitute a pulse plane, which is a two-dimensional matrix recording the photon detection states of all pixels at that moment.

[0056] Furthermore, the pulse sequences of several adjacent pixels are combined together to form pulse sub-arrays. These pulse sub-arrays can provide more detailed local photon arrival information, which is of great significance for analyzing the vibration characteristics of an object. Although the data of each pixel is single-bit quantized, when the sampling frequency of the pulse camera is high enough, it can capture high-speed visual phenomena that are difficult to record by traditional high-speed cameras. This high-sampling-rate pulse camera technology makes it possible to record the process of fast dynamic photon flow, bringing new measurement means to the field of visual vibration measurement.

[0057] In one embodiment of the present invention, the display 4 is used to display the image of the object to be measured. This function allows users to intuitively observe the vibration of the object to be measured. Preferably, users can manually or automatically select high-quality pixel blocks in the image, which have higher signal-to-noise ratio and more accurate vibration information in vibration measurement.

[0058] In addition, the speaker 5 is used to present the vibration signal in the form of sound in the embodiment of the present invention. The acoustic vibration signal processed by the signal processing module 4 is sent to the speaker 5 and converted into sound output. This enables users to not only visually observe the vibration effect, but also aurally perceive the vibration signal, providing a more comprehensive experience for vibration analysis. In this way, real-time reconstruction and multi-sensory display of the vibration signal can be achieved.

[0059] Second Embodiment

[0060] As Figure 5 shown, the second embodiment of the present invention provides a method for real-time reconstruction of vibration signals, which at least includes the following steps:

[0061] S1: Power on and start up;

[0062] S2: Convert the high-speed pulse array data captured by the pulse camera into a low-speed traditional image stream, and adjust the parameters so that the target area is clearly imaged on the display;

[0063] S3: Search for corner points and calibrate the central positions of the corner points;

[0064] S4: Determine whether the number of corner points meets the requirements; if the number of corner points is greater than or equal to a preset threshold (for example, 10, but not limited to this), then go to step S5; otherwise, return to step S2 to re-search for corner points by adjusting the target area and parameters;

[0065] S5: Generate pulse sub-arrays and organize them into a data stream;

[0066] S6: Reconstruct the acoustic vibration signal corresponding to the vibration of the object to be measured by using the information extracted from the data stream;

[0067] S7: Perform multi-channel signal merging and noise reduction processing, and integrate and optimize the acoustic vibration signals recovered from multiple pulse sub-arrays;

[0068] S8: Judge the signal-to-noise ratio of the acoustic vibration signal by the output signal of the speaker; if the signal-to-noise ratio meets the preset value, keep the voice output; otherwise, return to step S2.

[0069] In an embodiment of the present invention, step S2 involves processing the data captured by the pulsed camera to facilitate subsequent image display and vibration analysis. This step includes the following sub-steps:

[0070] S21: First, downsample the high-speed pulsed array data to reduce its speed.

[0071] Suppose the original sampling frequency of the pulsed camera is 200KHz, the resolution is 400×250, and the quantization bit width is 1 bit. Directly outputting these data requires an interface bandwidth as high as 2500MBsps. To reduce the bandwidth requirement of the data interface and the computing power required for image reconstruction, the embodiment of the present invention adopts the downsampling technology to reduce the sampling frequency to 1KHz, so that the data rate of the pulsed array is reduced to 12.5MBsps. Downsampling is a method of reducing the number of samples to lower the signal sampling frequency, which helps to reduce the data volume while retaining important signal features.

[0072] S22: Then, use the virtual exposure technology to convert the processed pulsed array data into a low-speed traditional image stream. Specifically, convert the pulsed array data into images with a frame rate of 24 frames per second and display them on the monitor. The virtual exposure technology simulates the exposure process of a traditional camera, and generates an image with appropriate brightness and contrast by accumulating pulsed signals within a certain time window.

[0073] S23: Finally, adjust the settings of the lens, such as aperture, focal length, and viewfinder range, according to parameters such as distance, lighting conditions, and the position of the acoustic vibration source. The purpose of these adjustments is to ensure that the target area can be clearly imaged on the monitor and as many corner points as possible are included in the image.

[0074] Through these sub-steps, step S2 not only reduces the burden of data transmission and processing, but also provides high-quality basic data for subsequent vibration signal analysis and image display. These processed image data will be used for the recovery of acoustic vibration signals and further signal processing to achieve real-time reconstruction of vibration signals.

[0075] A corner point is a point where two edges in an image meet. The change in pixel amplitude caused by object vibration is more obvious at these points. Therefore, corner points are crucial for the analysis of vibration signals. In an embodiment of the present invention, step S3 focuses on the detection and screening of corner points to facilitate subsequent vibration signal analysis. This step includes the following sub-steps:

[0076] S31: First, for each frame of the reconstructed traditional image, apply the Harris corner detection algorithm to identify the corner points in the image. The Harris corner detection is an algorithm widely used in image processing to find the points where edges in the image meet, that is, corner points. After identifying the corner points, determine the pixel position at the center of each corner point to provide accurate coordinate information for subsequent analysis.

[0077] S32: Then, calculate the average of the detected corner point center positions in 24 consecutive frames of images (i.e., the images captured within 1 second) to obtain a more stable and reliable corner point center position. These averaged corner point center positions will be marked on the display, enabling the user to visually see the positions of all corner points.

[0078] S33: Then, according to the position of the acoustic vibration source and the position of the corner points on the target object, screen the corner points according to preset criteria. These preset criteria include but are not limited to: 1) The candidate corner points should be close to the acoustic vibration source to improve the detection accuracy of vibration signals; 2) The distance between corner points should be greater than or equal to 40 pixels to ensure that the noises are uncorrelated and improve the signal clarity; 3) The distance between the corner point center and the image edge should be greater than or equal to 10 pixels to facilitate the extraction and processing of subsequent pulse sub-arrays.

[0079] S34: Finally, record the number of candidate corner points after screening and their center positions. This information is crucial for generating the pulse sub-array data stream because it will be used in the subsequent acoustic vibration signal recovery process.

[0080] Through these sub-steps, step S3 ensures the accurate detection and screening of corner points, providing high-quality basic data for the recovery of vibration signals. The processed corner point information will be used to generate the pulse sub-array data stream, thereby realizing the real-time reconstruction of vibration signals.

[0081] In an embodiment of the present invention, step S5 specifically includes the following sub-steps:

[0082] S51: Refer to Figure 6, first, according to the corner center positions determined in step S3, the corresponding pixel positions are accurately retrieved in the pulse array. Based on these corner positions, pulse sub-arrays with a size of 3×3×N (where N is a positive integer representing the length of the time series) are generated. These pulse sub-arrays contain the vibration information associated with the corners and are the basis for subsequent acoustic-vibration signal recovery. Each sub-array consists of 3×3 pixel blocks, and these pixel blocks are continuously recorded in the time series, forming a three-dimensional data structure containing vibration information.

[0083] S52: Next, a data stream composed of multiple (e.g., 10, but not limited to) such pulse sub-arrays is output in parallel. This means that the pulse sub-arrays generated from multiple different corner positions simultaneously will be output together. This parallel processing method can significantly improve the data processing efficiency and allows the system to analyze multiple vibration sources simultaneously. Each data stream of the pulse sub-arrays contains the vibration information extracted from the corresponding corner positions, and these data streams will be used for the recovery of the acoustic-vibration signal.

[0084] Through step S5, the sub-arrays related to vibration can be extracted from the original high-speed pulse array data, and these sub-arrays are organized into a data stream, providing preparation for the recovery of the acoustic-vibration signal. This step is one of the key links in realizing the real-time reconstruction of the vibration signal, which enables the high-speed data captured by the pulse camera to be converted into a format that can be used for vibration analysis.

[0085] In an embodiment of the present invention, step S6 is a key link in the process of acoustic-vibration signal recovery, specifically including the following sub-steps:

[0086] S61: As Figure 7 shown, the pulse sequences output by each pixel in the pulse sub-array are remapped. In a preferred embodiment of the present invention, the pulse sub-array consists of 9 pulse sequences, and each sequence corresponds to a pixel in the sub-array. During the mapping process, the element "0" in the original pulse sequence is converted to "-1", and the element "1" is converted to "+1", and remapped into a new pulse sequence where i represents the pixel serial number, used to identify a single pixel in the pulse sub-array; j represents the sub-array serial number, used to identify different pulse sub-arrays; n represents the element serial number in the pulse sequence output by each pixel. In the data captured by the pulse camera, each pixel generates a pulse sequence, and this sequence changes over time. n is used to identify a specific time point or pulse element in these sequences. The purpose of this mapping rule is to facilitate subsequent signal processing, making the positive and negative changes of the signal more obvious, thus helping to extract vibration information.

[0087] S62: Generate a difference sequence. The difference sequence is obtained by calculating the differences between adjacent elements of the mapped pulse sequence, and its formula is where is the mapped pulse sequence, is the difference sequence. The elements of the difference sequence include 2, 0, and -2. This kind of difference processing helps to eliminate the DC signal in the pixel intensity, making the signal more concentrated on the vibration changes.

[0088] S63: As Figure 8 shown, convert the 1-bit signal of high-speed sampling (e.g., 200KHz) to a 16-bit signal of low-speed sampling (e.g., 10KHz) through a cascaded integrator-comb (CIC) filter. The CIC filter is an efficient digital signal processing filter that can effectively reduce the sampling rate without introducing additional phase distortion. Subsequently, suppress the high-frequency interference signal through a finite impulse response (FIR) low-pass filter with a cut-off frequency of 3KHz. Due to its linear phase characteristic, the FIR filter can maintain the waveform characteristic of the signal while removing high-frequency noise. After these processes, the final output signal is the acoustic-vibration signal after noise reduction and speed reduction processing, providing a high-quality data basis for the further analysis and application of the acoustic-vibration signal.

[0089] It should be noted that the cascaded integrator-comb (CIC) filter is composed of a cascade of an integrator and a comb filter. The design of this filter is particularly suitable for hardware implementation because its structure is simple, only containing adders and delay elements without involving multiplication operations. As Figure 9 shown, the cascaded integrator-comb filter has a decimation factor of 4, which means it reduces the sampling rate of the input data to one-fourth of the original, and the order of the filter is 5, which determines the response characteristic of the filter. To handle the possible integrator overflow problem, the data bit width in the filter is set to 32 bits.

[0090] In an embodiment of the present invention, each sub-array is a 1-bit quantization unit composed of a 3×3 pixel block on the array plane. Considering that the cascaded integrator-comb filter can handle a high-speed sampling frequency of up to 200kHz, the data rate of each sub-array is 225kBsps. When processing 10 sub-arrays simultaneously, the total data rate reaches 2550kBsps. This design significantly reduces the demand for the data interface bandwidth, thus more efficiently processing the high-speed pulse array data from the pulse camera and providing a technical basis for realizing the real-time reconstruction of vibration signals.

[0091] S64: Recover the acoustic-vibration signal through weighted summation.

[0092] where, the formula for recovering the acoustic-vibration signal through weighted summation is:

[0093]

[0094] wherein, is the signal output after the restoration processing of the i-th pixel in the j-th pulse sub-array; is the weight for combining the signals of pixel i, and is determined according to the output signal-to-noise ratio.

[0095] Through these sub-steps of step S6, the present invention can restore a clear acoustic vibration signal from the pulse sub-array, providing accurate data support for vibration analysis and sound reproduction. This process is a key link in realizing the real-time reconstruction of vibration signals, enabling the high-speed data captured by the pulse camera to be converted into acoustic vibration signals that can be used for vibration analysis.

[0096] In an embodiment of the present invention, step S7 includes the following sub-steps:

[0097] S71: Synthesize multi-channel data into a single acoustic vibration signal according to the weights; wherein, the formula for synthesizing a single acoustic vibration signal S(m) is:

[0098]

[0099] wherein, w j is the weight corresponding to channel j.

[0100] It should be noted that w j is determined by the signal-to-noise ratio SNR j of the sub-acoustic vibration signal, and its formula is:

[0101]

[0102] wherein, Th is the signal-to-noise ratio threshold, and this threshold is determined according to experimental data and / or experience.

[0103] S72: Use spectral subtraction to perform noise reduction processing on the synthesized acoustic vibration signal S(m).

[0104] It should be noted that spectral subtraction is a noise suppression technique widely used in the field of digital signal processing, especially in speech enhancement applications. The core principle of this technique is to estimate the spectrum of the noise and then subtract this estimated value from the spectrum of the speech signal containing the noise to achieve noise suppression. The steps to implement spectral subtraction include: first, signal acquisition to obtain the original speech signal; then, perform Fourier transform on the acquired signal to convert it to the frequency domain; next, perform noise estimation to identify and quantify the noise components in the frequency domain; then, perform spectral subtraction operation to subtract the estimated noise spectrum from the spectrum of the speech signal; finally, apply the inverse Fourier transform to convert the processed signal back to the time domain to obtain the denoised speech signal. Through this series of steps, spectral subtraction can effectively reduce the noise level in the speech signal and improve the clarity and intelligibility of the speech.

[0105] It should be noted that the above-mentioned multiple embodiments are only examples. The technical solutions of each embodiment can be combined, and all are within the protection scope of the present invention.

[0106] The above has described in detail the device and method for real-time reconstruction of vibration signals provided by the present invention. For those of ordinary skill in the art, any obvious changes made to it without departing from the essence of the present invention will constitute an infringement of the patent right of the present invention and will bear corresponding legal responsibilities.

Claims

1. A device for real-time reconstruction of vibration signals, characterized in that include: A pulse camera, used for collecting image data and sending the collected image data to the data interface module; A data interface module, used for receiving high-speed pulse array data sent by the pulse camera, and sending the high-speed pulse array data to the signal processing module; A display, used for receiving and displaying a low-speed conventional image stream sent by the signal processing module; A signal processing module, used for receiving the high-speed pulse array data sent by the data interface module, processing the data, generating a low-speed traditional image stream and sending it to the display, and restoring the acoustic vibration signal of the measured object and sending it to the speaker; The speaker is used to receive and play the voice signal of the measured object sent by the signal processing module.

2. The device according to claim 1, characterized in that The signal processing module comprises: A data acquisition unit, used for receiving the high-speed pulse array data sent by the data interface module, and sending the data to the data interface unit; A data interface unit, used for receiving high-speed pulse array data sent by the data acquisition unit, sending the data to the pixel block screening unit, sending the sub-array stream selected by the corner point search to the acoustic vibration signal recovery unit, and receiving the position information of the effective vibration pixel sent by the pixel block screening unit; A pixel block screening unit, used for converting high-speed pulse array data into low-speed pulse array data by down-sampling, and converting the low-speed pulse array data into a low-speed traditional image stream to be sent to a display; An acoustic vibration signal recovery unit, used to collect multiple sub-array streams, recover corresponding acoustic vibration signals, and send the recovered acoustic vibration signals to a signal post-processing unit; The signal post-processing unit is used to synthesize multiple sound vibration signals into one voice signal and send the synthesized voice signal to the speaker.

3. The device according to claim 2, characterized in that The pixel block screening unit performs down-sampling processing on the high-speed pulse array data to reduce the sampling frequency; and adopts the virtual exposure technology to convert the processed pulse array data into a low-speed traditional image stream.

4. A method for real-time reconstruction of a vibration signal, implemented based on the device according to any one of claims 1 to 3, characterized in that The steps include: S1: Power on and start; S2: Convert the high-speed pulse array data captured by the pulse camera into a low-speed traditional image stream and adjust the parameters so that the target area is clearly imaged on the display; S3: Search for corner points and calibrate the center positions of the corner points; S4: Determine whether the number of corner points meets the requirement; if the number of corner points is greater than or equal to the preset threshold, proceed to step S5; Otherwise, return to step S2 and search for corner points again by adjusting the target area and parameters; S5: Generate pulse subarrays and organize them into data streams; S6: reconstructing an acoustic vibration signal corresponding to the vibration of the measured object using the information extracted from the data stream; S7: Perform multi-channel signal merging and noise reduction processing to integrate and optimize the acoustic vibration signals recovered from multiple pulse sub-arrays; S8: Determine the signal-to-noise ratio of the acoustic vibration signal through the speaker output signal; if the signal-to-noise ratio meets the preset value, keep the voice output; otherwise return to step S2.

5. The method according to claim 4, characterized in that The step S2 includes the following sub-steps: S21: Slowing down the high-speed pulse array data by decimation; S22: converting pulse array data into a conventional image with a preset frame rate using virtual exposure technology and displaying it on a monitor; S23: adjusting the aperture, focal length and framing range of the lens according to the distance, lighting conditions and acoustic vibration source position parameters, so that the target area is clearly imaged on the display and as many corner points as possible are included in the image.

6. The method according to claim 4, characterized in that The step S3 includes the following sub-steps: S31: searching corner points for each frame of the reconstructed traditional image using the Harris corner detection algorithm, and determining the pixel position of the center of the corner point; S32: averaging the center positions of the corner points retrieved from the multiple frames of images to obtain the averaged center positions of the corner points, and marking the positions of all the corner points on the display; S33: screening the corner points according to a preset criterion based on the position of the acoustic vibration source and the position of the corner points on the target object; S34: Record the number of candidate corner points and the center positions of the corner points.

7. The method according to claim 6, characterized in that The preset criteria include but are not limited to: 1) the candidate corner points should be close to the acoustic vibration source to improve the detection accuracy of the vibration signal; 2) the distance between the corner points should be greater than or equal to 40 pixels to ensure that the noise is uncorrelated; 3) The distance between the center of the corner point and the edge of the image should be greater than or equal to 10 pixels to facilitate the subsequent extraction and processing of the pulse subarray.

8. The method according to claim 4, characterized in that The step S5 comprises the following sub-steps: S51: Retrieving the corresponding pixel position in the pulse array according to the center position of the corner point, and generating a pulse subarray of a preset size; S52: parallel output of a data stream organized by multiple pulse sub-arrays.

9. The method according to claim 4, characterized in that The step S6 comprises the following sub-steps: S61: Pulse sequence output for each pixel in the pulse subarray Remap to generate new pulse train The mapping rule is: The element "0" in is mapped to "-1", The element "1" in is mapped to "+1"; S62: Generate a differential sequence by calculating the difference between adjacent mapped pulse sequence elements; S63: The high-speed sampled 1-bit signal is converted into a low-speed sampled 16-bit signal through a cascaded integrator comb filter, and the high-frequency interference signal is suppressed through a finite impulse response low-pass filter to obtain an output signal S64: Restore the acoustic vibration signal by weighted summation using the following formula: in, is the signal output after the i-th pixel is restored and processed in the j-th pulse subarray; is the weight of the pixel i signal merging, according to The signal-to-noise ratio of the output is determined.

10. The method according to claim 4, characterized in that The step S7 includes the following sub-steps: S71: synthesize the multi-channel data into one sound vibration signal according to the weights; wherein the formula for synthesizing one sound vibration signal S(m) is: The w j is the weight corresponding to channel j, and the signal-to-noise ratio SNR of the sub-acoustic vibration signal j Sure; S72: Use spectral subtraction to perform noise reduction processing on the synthesized acoustic vibration signal S(m).

Images