Omnidirectional vibration monitoring system based on RGB image and optical fiber sensing

Abstract

The invention discloses an omni-directional vibration monitoring system based on RGB images and optical fiber sensing, and the system comprises an RGB lamp array which is disposed on an object to be measured; the receiver is fixed in position and surrounds the RGB lamp array, and when the RGB lamp array moves along with the to-be-measured object, a light field in the receiver changes; the three optical fibers are respectively fixed at three direction positions of the receiver and are used for collecting light field information of three channels of RGB (Red, Green, Blue); the coupler is used for coupling information of the three optical fibers and is connected to the camera through one optical fiber; the camera is used for collecting an RGB image; and the upper computer is used for acquiring vibration information of the to-be-measured object according to the acquired RGB image. According to the invention, a monitoring scheme that the optical fiber is used as a probe and cooperates with the RGB image to measure vibration is established, and the monitoring scheme is applied to multi-point vibration monitoring of a large object, so that the cost is greatly reduced. The method can be widely applied to the field of vibration monitoring.

Description

Technical Field

[0001] The present invention relates to the field of vibration monitoring, and particularly to an omnidirectional vibration monitoring system based on RGB images and fiber optic sensing. Background Art

[0002] Vibration monitoring has important applications in the industrial and scientific research fields. In the field of vibration monitoring, fiber optic vibration sensors have gradually become a research hotspot due to their unique advantages. And with the progress of machine vision and image recognition technologies, there are also many solutions for directly completing vibration monitoring using image recognition. The current implementation solutions of vibration sensors mainly include: 1) Distributed fiber optic vibration sensors, which use optical fibers as sensing media to achieve continuous monitoring of vibrations. Such sensors detect vibrations by measuring changes in the propagation characteristics of light in the optical fiber, and have characteristics such as distributed measurement, high sensitivity, and anti-electromagnetic interference. 2) Fiber optic vibration sensors based on Mach-Zehnder interferometers, which detect vibrations by measuring changes in the optical path difference between the two arms of the interferometer. 3) Distributed vibration sensors based on OTDR technology, which detect vibrations by measuring the time delay and intensity changes of the backward scattered light in the optical fiber. 4) Vibration monitoring based on image recognition and motion tracking, which can accurately monitor and analyze the vibration behavior of downleads under the action of wind through high-precision motion tracking technology.

[0003] Although fiber optic vibration sensors are now becoming more and more mature and there have been significant improvements in aspects such as accuracy, there are still many problems to be solved in the current solutions; in terms of optical fibers, most measurements are made by changing the propagation characteristics of light in the optical fiber or using the diffraction and interference characteristics of light, which have great limitations in large-scale deployment and cost control; while in terms of image recognition, there are relatively high requirements for the lighting conditions during measurement and the front-end embedded computing power.

[0004] Taking the four technologies described above as an example, the deployment of distributed fiber-optic vibration sensors requires a high initial investment, especially in large-scale deployments. Mach-Zehnder interferometers are highly sensitive to environmental changes (such as temperature and pressure), which can affect measurement stability and accuracy. Furthermore, interferometer equipment is generally expensive, limiting their widespread adoption in large-scale applications. OTDR technology, when used for long-distance monitoring, can suffer from signal attenuation within the optical fiber, which can affect sensitivity and accuracy. Furthermore, in certain environments or with certain materials, OTDR technology may not provide sufficient measurement accuracy. Image recognition technology, while showing great potential, currently has limited application scenarios. It is significantly affected by lighting conditions, obstructions, and background noise, which can reduce monitoring accuracy. Furthermore, high-precision motion tracking and image processing typically require high computing resources, increasing system operating costs. In scenarios requiring high-speed or real-time monitoring, image processing and analysis can experience delays, impacting real-time monitoring. Summary of the Invention

[0005] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the present invention aims to provide an omnidirectional vibration monitoring system based on RGB images and fiber optic sensing.

[0006] The technical solution adopted by the present invention is:

[0007] An omnidirectional vibration monitoring system based on RGB images and fiber optic sensing, comprising:

[0008] RGB light array, set on the object to be tested;

[0009] The receiver is fixed in position and surrounds the RGB light array. When the RGB light array moves with the object to be measured, the light field inside the receiver will change.

[0010] Three optical fibers are fixed at three directions of the receiver to collect light field information of the three RGB channels;

[0011] A coupler is used to couple the information of the three optical fibers and connect it to the camera through one optical fiber;

[0012] Camera, used to collect RGB images;

[0013] The host computer is used to obtain the vibration information of the object to be measured based on the obtained RGB image.

[0014] Furthermore, the three viewing angles of the three optical fibers are perpendicular to each other, and the three RGB channels each correspond to one degree of freedom. Light field calibration is required before monitoring the object to be measured.

[0015] Furthermore, obtaining vibration information of the object to be measured based on the obtained RGB image includes:

[0016] By inversely solving the RGB image, the light field information of three channels is obtained;

[0017] The three RGB color channels are processed separately to obtain the brightness curves of each channel over time. These curves reflect the changes in light intensity during the vibration process.

[0018] First, the pixel values of each channel are averaged to obtain the average brightness value of the channel. Then, the brightness change between consecutive frames is calculated to track the light intensity fluctuation during the vibration process.

[0019] By analyzing the processed data, the characteristics of the vibration event are identified, and the vibration information of the object to be measured is obtained based on the identified characteristics; wherein the identified characteristics include the frequency, amplitude and possible vibration mode of the vibration.

[0020] Furthermore, the host computer is specifically used for:

[0021] The camera is called through Matlab programming to capture RGB color change videos caused by vibration signals. Next, frames are extracted from the video and converted into a three-dimensional matrix, and data from the three RGB channels is extracted separately. A filter is applied to smooth the data to clearly observe the periodicity and amplitude of the signal when plotting. Finally, the processed data is analyzed and visualized to verify the accuracy of the experimental results.

[0022] Furthermore, the analysis and visualization of the processed data includes:

[0023] The frequency is calculated using Fourier analysis, and the three-dimensional vibration information of the object to be measured is solved using the frequency data in three directions.

[0024] Furthermore, if a USB-connected camera is used, according to the experimental setup design, ensure that the fiber reading area matches the camera's field of view, and calculate the average brightness within this area; then, normalize the brightness data to eliminate the influence of lighting.

[0025] The beneficial effect of the present invention is that the present invention establishes a monitoring solution that uses optical fiber as a probe and coordinates RGB images to measure vibration, and applies it to multi-point vibration monitoring of large objects, greatly reducing costs.

[0026] Compared with traditional visual methods for measuring vibration, each point to be measured requires a monitoring camera. The exposed monitoring camera not only has high costs and is easily damaged, but also records an extremely large amount of data. With the monitoring solution of the present invention, only one camera is used to complete the vibration monitoring that previously required hundreds of cameras. When facing multi-point vibration, the advantages of "optical fiber + camera" can be fully utilized. Only a very low resolution is required to analyze and solve the vibration information in the optical fiber, and the vibration situation at that point can be quickly obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following provides an introduction to the relevant technical solution drawings in the embodiments of the present invention or the prior art. It should be understood that the drawings in the following introduction are only for conveniently and clearly expressing some embodiments of the technical solutions in the present invention. For those skilled in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0028] Figure 1 is the basic structure diagram of an omnidirectional vibration monitoring system based on RGB images and fiber optic sensing;

[0029] Figure 2 is a schematic diagram of the changes in RGB-LED arrays of each generation;

[0030] Figure 3 is a schematic diagram of fiber optic coupling related parameters;

[0031] Figure 4 is a signal collection and processing flow chart;

[0032] Figure 5 is a Fourier analysis fundamental wave result diagram;

[0033] Figure 6 is a schematic diagram showing the experimental devices;

[0034] Figure 7 is a diagram showing the light field distribution and the LED measurement results;

[0035] Figure 8 is a schematic diagram of the vibration displacement calibration results;

[0036] Figure 9 is a schematic diagram of the brightness change area on the optical fiber;

[0037] Figure 10 is a schematic diagram showing the pre-experiment results;

[0038] Figure 11 is the Fourier spectrum diagram in the pre-experiment;

[0039] Figure 12It is a schematic diagram showing relevant results of beat frequency simulation;

[0040] Figure 13 This is a schematic diagram showing the optical fiber device used in the formal experiment;

[0041] Figure 14 It is a diagram showing the change of relative light intensity of RGB channels over time and the curve after the blue light is turned off;

[0042] Figure 15 This is the comparison result of the R, G, and B channel measurement frequency and the signal source frequency at 20Hz;

[0043] Figure 16 It is the Fourier spectrum diagram of the experimental results at a signal source frequency of 19 Hz and 9 Hz;

[0044] Figure 17 It is the amplitude residual graph of green light, blue light and red light at 19Hz and 800mV;

[0045] Figure 18 It is a schematic diagram of the spatial motion trajectory of the measurement data synthesis;

[0046] Figure 19 It is a schematic diagram of spatial vibration measurement and a comparison diagram of the curve synthesized by the measured values and the real space trajectory. DETAILED DESCRIPTION

[0047] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. For the step numbers in the following embodiments, they are provided only for the convenience of explanation and are not intended to limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0048] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application. The singular forms of "a", "said", and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise clearly defined, words such as setting, installing, and connecting should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

[0049] In the description of the present application, it should be understood that for the orientation description, such as the orientation or positional relationship indicated by up, down, front, back, left, right, etc., it is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or component referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present application.

[0050] In the description of the present application, the meaning of several is one or more, the meaning of multiple is more than two, greater than, less than, exceeding, etc. are understood as not including the present number, above, below, within, etc. are understood as including the present number. If the first and second are described, it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or implicitly indicating the sequence relationship of the indicated technical features.

[0051] In the description of the present application, "and / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the front and back associated objects.

[0052] In view of the existing technical problems, the present embodiment provides an omnidirectional vibration monitoring system based on RGB images and fiber optic sensing, which will be described below in conjunction with the drawings and specific embodiments.

[0053] As Figure 1 shown, Figure 1 The basic structure of this monitoring solution is shown. When the RGB light array attached to the vibrating object moves with the vibrating object, the light field in the receiver will change. The three optical fibers fixed in the receiver respectively collect the color and brightness information of the light fields in the RGB three channels and transmit them to the camera. After the camera takes a video with color and brightness changes and analyzes it, the vibration result can finally be solved.

[0054] In some embodiments, referring to Figure 2 , Figure 2 shows the iteration of the RGB-LED light array. Figure 2 In (a), it is the initial generation light array. The leads of the light-emitting diodes are relatively long. When attached to the vibrating object and vibrating, it will shake, and it needs an external battery for power supply, with a relatively large mass, which affects the accuracy of experimental data. Figure 2 In (b), it is the second-generation light array. The light-emitting diodes are integrated in the cube, reducing the mass. Different wavelength filters are used to filter out the RGB three-color lights, and magnetic control switches and low-frequency flashing are realized, but its brightness is relatively low. Figure 2Figure (c) shows the third-generation light array, which not only improves the brightness but also enables replaceable batteries, greatly improving the experimental detection scheme. Figure (d) is the 3D modeling of the light array cube.

[0055] The working principle is as follows: Since the three vibration signals propagate independently in the RGB primary color channels, they can be transmitted through a single optical fiber. These three signals can then be deconstructed from the RGB image, resulting in a vibration monitoring device with three measurable degrees of freedom. The following section will focus on explaining the principles involved in the solution.

[0056] In the experiment, a platform that can output three-dimensional vibrations can be built by superimposing an oscillator on a horizontal shaker.

[0057] (1) Fiber optic receiving principle

[0058] Fiber optic transmission is based on the propagation characteristics of light in an optical fiber. Single-mode fiber is chosen for its low loss and high bandwidth potential. In this example, a three-in-one coupling fiber was constructed, using three 1.4 mm and one 3 mm PMMA optical fibers, coupled by direct end-to-end docking. This coupling method reduces the space cost associated with optical couplers.

[0059] Figure 3 The relevant parameters and effects of fiber coupling are shown. Figure 3 (a) is the end face of a PMMA optical fiber with a diameter of 1.4 mm. Figure 3 (b) is the end face of a PMMA optical fiber with a diameter of 3 mm. Figure 3 (c) is the effect diagram of four optical fiber coupling. Figure 3 (d) shows the effect after fiber coupling is completed and some fiber parameters.

[0060] Four optical fibers were directly fused together using the end-face coupling method, and the outer layer was protected with EVA glue. EVA glue is transparent and light-permeable, and is plastic at high temperatures. Therefore, the situation at the optical fiber coupling point can be visually seen through this protective glue, and any disconnection or other problems can be discovered and corrected in a timely manner.

[0061] (2) Sensor reception and signal processing

[0062] A digital image can be represented by a two-dimensional array, where each element corresponds to the RGB value of a pixel. An image sensor, such as a camera, captures the RGB light intensity of a scene and converts it into a digital image. The average brightness of an image can be calculated by averaging the RGB values of all pixels:

[0063]

[0064] Where, L 平均 is the average brightness of the image, N is the total number of pixels in the image, R i ,G i ,B i The image sensor used in the experiment of this embodiment is Sony IMX600y sensor, which is a CMOS image sensor.

[0065] In this experiment, the camera is first called through Matlab programming to collect RGB color change video caused by vibration signals; then, frames are extracted from the video and converted into a three-dimensional matrix, and data of the three RGB channels are extracted respectively; if a USB-connected camera is used, the experimental device design will ensure that the fiber reading area matches the camera's field of view, and the average brightness in the area is calculated; then, the brightness data is normalized to eliminate the influence of lighting; further, a filter is applied to smooth the data so that the periodicity and amplitude of the signal can be clearly observed when drawing; finally, the processed data is analyzed and visualized to verify the accuracy of the experimental results. The signal processing flow is as follows: Figure 4 shown.

[0066] Furthermore, in order to remove high-frequency noise from the collected signal, a Butterworth low-pass filter was used in the experiment. The frequency and amplitude of the vibration signal can be analyzed by signal processing techniques such as Fourier transform:

[0067]

[0068] Where P(f) is the spectrum of the vibration signal, v(t) is the vibration signal in the time domain, and f is the frequency. Figure 5 During vibration, since the monitored light intensity signal is not 0 at the origin, there will be a strong DC component. Therefore, the Fourier analysis will have an obvious peak at f=0. This is caused by the average DC component brought by the external field brightness. Therefore, this component is not discussed in the spectrum analysis, and only the frequency value of the second peak height is considered as the fundamental frequency.

[0069] like Figure 5 As shown in the Fourier analysis results, the red dotted line points to the peak value caused by the DC component, and the data mark is the frequency value of the second peak height as the fundamental frequency, such as Figure 5 The vibration frequency shown is 34Hz, with an error of only 0.3%.

[0070] (3) Beat frequency measurement and accuracy improvement

[0071] Beat frequency is usually generated by two wave sources with very close frequencies. When they are superimposed, periodic intensity variations are formed. When two waves have a small frequency difference, and they propagate in the same direction, have the same amplitude, and the same deflection direction, according to the superposition principle:

[0072]

[0073] where E1 = E0cosω0t and E2 = cos(ω0 + Δω)t. These are two sine wave signals, and the envelope amplitude variation frequency Δω is called the beat frequency. Simply put, by superimposing two sine signals, given the vibration frequency of one signal, we can measure the other signal to be measured through the beat frequency.

[0074] The condition for the two waves to complete superposition is that they must be in the same medium and propagate in the same way. In this experiment, the experimental group generated beat frequency through the flashing of the light source and the vibration signal to be measured. Although they do not have the same propagation mode, after visual capture, they can both be converted into digital signals. After relevant processing of the signals, the measurable beat frequency can be obtained.

[0075] (4) Preliminary experiment and problem correction

[0076] Figure 6 shows the devices used in the preliminary experiment. Figure 6 In (a) is the rendering diagram of the device modeling. Figure 6 In (b) is the 3D printed device. Figure 6 In (c) is the flat TPU optical fiber. Figure 6 In (d) is the RGB lamp array, used to establish the light field. The different lamps are separated by opaque sandpaper to prevent interference between light fields of different colors.

[0077] The preliminary experiment will use the above devices for demonstration. When measuring three-direction vibration, the flat TPU optical fiber is used to replace the PMMA optical fiber for one-to-three optical signal transmission, simplifying the steps of fiber coupling production. However, the inhomogeneity inside the TPU optical fiber brings large signal attenuation inside the fiber, and this method cannot be used for long-distance transmission. Therefore, it is only used in the preliminary experiment.

[0078] 4.1) Light field calibration

[0079] In the previous discussion, it was mentioned that each of the RGB three channels corresponds to a degree of freedom. However, since the light source for establishing the light field is an LED, the excited light field is easily reminiscent of a symmetric light field that conforms to the inverse square law in space. Coincidentally, the experiment utilized the symmetric relationship of the light field in space. When the optical fiber moves two-dimensionally within the light field, the color and light intensity changes caused by the motion component orthogonal to the calibration direction can be eliminated by the symmetric light field, which lays the foundation for using the three channels to select three independent degrees of freedom to solve the full-directional spatial vibration.

[0080] Figure 7 Some data on light field distribution and LED measurement are shown. Figure 7 In (a), it is the LED wavelength curve measured by a grating spectrometer. Figure 7 In (b), it is a schematic diagram of the motion range of the optical fiber in the symmetric light field. Figure 7 In (c), it is the light intensity distribution surface measured by a Sony IMX600y sensor under different surfaces and different channels. Since no light-shielding paper was added when measuring the light intensity distribution of each surface, there is some light intensity noise of other channels under a single surface. However, as Figure 7 shown, the noise light intensity has an order-of-magnitude difference from the light intensity of the main channel on that surface. Therefore, the noise brought by the LED on the other surfaces to the light emission of the main channel surface can be ignored.

[0081] In addition, in order to reduce the amount of calculation, this embodiment quantifies the mathematical error of using linear approximation to inversely solve the displacement. When discussing the local linearization problem of the physical quantity described by the inverse square law - light intensity, its mathematical expression form will be considered first where k is the proportionality constant. In order to perform linear approximation of this physical quantity in a local area, the method of Taylor series expansion can be adopted.

[0082] 4.2) Parameter measurement and calibration

[0083] 1) Vibration displacement calibration

[0084] After completing the light field measurement, the amplitude of the oscillator was calibrated. Since the turning radius of the horizontal shaker used to generate x - y direction vibration is a fixed value of 1 mm, the product parameter information is directly cited here as the standard amplitude information. For the amplitude calibration of the oscillator, this embodiment used a camera to take high-definition videos and tracked the amplitude in the Tracker software.

[0085] Figure 8 It is part of the data for calibrating the oscillator amplitude using the Tracker software. Figure 8 In (a) and (d), they are the calibration curves when the amplitude of the signal generator is 10 v. Figure 8 In (b) and (e), they are the calibration curves when the amplitude is 5 v. Figure 8In (c) and (f), they are the calibration curves at an amplitude of 2v. The detailed data is shown in Table 1 below.

[0086] Table 1 Amplitude Calibration Data

[0087]

[0088] It can be seen that the amplitude size changes basically linearly with the output amplitude, and there is a good fit in multiple groups of measurement data. Therefore, the following relationship can be obtained:

[0089]

[0090] Among them, L is the amplitude size, with the unit of mm, and U is the output amplitude size of the signal generator, with the unit of V.

[0091] 2) Measurement Accuracy Calculation

[0092] In this monitoring scheme, the measurement accuracy is closely related to parameters such as the fiber optic probe diameter and the number of pixels occupied by the fiber optic. Here, a method for calculating the accuracy is given taking the Sony IMX600y sensor and the 1.4mm diameter fiber optic probe used in this experiment as an example.

[0093] Since the video image specifications output by the IMX600y sensor are COMP8, COMP8 means that each pixel uses 8 bits to represent, that is, the brightness information of each pixel is composed of 8 bits, that is, each pixel can present 3×2 8 color information. 3 is the number of channels. Here, only a single channel is taken for analysis - each pixel can represent 256 different gray levels. In the experimental process, when processing the image brightness, the method of taking the average brightness of a certain area is used. Therefore, when the number of pixels is increased, the measurement accuracy will also be significantly improved. Taking two pixels as an example, theoretically there may be 256 2 brightness situations between the two pixels. However, considering that the spatial distance between pixels is extremely small and the brightness difference is small (the sudden brightness change caused by occlusion is not considered, and only the brightness change within the fiber optic range is considered here), there should be a brightness difference threshold between the two pixels. The maximum threshold measured in this experiment is 5, that is, the maximum brightness difference between the two pixels is 5 (the maximum brightness difference will increase as the number of pixels within the fiber optic range decreases). At the same time, in this experiment, a 500×500 area is defined for average value calculation. Therefore, the number of all possible average brightness values within this area can be obtained as:

[0094] N = 256×5 500

[0095] This value is obviously too large. Due to the spatial scale of the optical fiber, it cannot bring about a change in the brightness of a single pixel at each point. In actual measurement, within the field of view, the optical fiber does not change individually for each pixel. Pixels exhibit a certain degree of entanglement within a certain area, which can be roughly divided into 5 significantly changing regions.

[0096] Figure 9 What is shown is the regional division of the brightness change on the optical fiber. Based on the overall change in the brightness of the optical fiber within the field of view, the edges (regions 1, 3, 4, and 5) and the center (region 2) of the optical fiber exhibit different brightness change characteristics locally. Therefore, the number of all possible average brightness values within this region is:

[0097] N = 256 × 5 5 = 8 × 10 5

[0098] The minimum precision of the relative light intensity (after normalization) should be:

[0099]

[0100] According to the amplitude calibration data, the minimum precision of the amplitude measurement can be obtained as:

[0101] A Δ = 1.25 × 10 -5 mm

[0102] Since the precision of the vibration generating device can only reach the order of 10 -3 mm, the precision of the amplitude measurement can be set as:

[0103] A Δreal = 0.001mm

[0104] The measurement precision of the frequency is given by the Nyquist sampling theorem. The basic form of the Nyquist sampling theorem can be expressed as:

[0105] f sampling ≥ 2f sample

[0106] Among them, f sampling is the sampling rate, and f sample is the signal rate to be sampled. In the design of this experiment, the IMX600y can shoot videos at a maximum of 7680 FPS / s, and can measure vibration signals with a maximum frequency of 3840 Hz.

[0107] 4.3) Pre-experiment and Calibration Result Display

[0108] See Figure 10 , Figure 10Figure (a) shows the process of calculating the average brightness of different channels and the evolution of the average brightness over time. Figure 10 Figure (b) shows the change curves of the brightness of optical fibers in different channels when a single perturbation is applied three times. The second figure corresponds to Figure 10 the motion in Figure (a).

[0109] By processing the three RGB color channels separately, the brightness curves of each channel over time can be obtained, as shown in Figure 10 Figure (b). These curves reflect the change in light intensity during the vibration process and are an important basis for analyzing vibration characteristics. During the image processing process, first, the pixel values of each channel are averaged to obtain the average brightness value of that channel. Subsequently, by calculating the brightness change between consecutive frames, the light intensity fluctuations during the vibration process can be traced. By applying a Butterworth low-pass filter, high-frequency interference caused by environmental noise or sensor noise is further eliminated, ensuring the stability and reliability of the signal. Finally, by analyzing the processed data, the characteristics of vibration events, including the vibration frequency, amplitude, and possible vibration modes, can be accurately identified, providing strong data support for vibration monitoring.

[0110] In addition, in the experiment, an oscillator was also used to measure one-dimensional vibration data from 35 Hz 0.3 mm to 35 Hz 1 mm. The data is shown in Table 2:

[0111] Table 2 Peak-to-peak data of relative brightness

[0112]

[0113] Figure 11 Figures (a) and (b) show the spectral analysis curves at a vibration frequency of 35 Hz, and the errors are all within 5%. After pre-experiment and calibration, the formal experiment was started.

[0114] 4.4) Beat frequency measurement experiment and simulation

[0115] To verify the feasibility of beat frequency measurement, a simulation of using beat frequency to complete vibration measurement was carried out. A sine wave with a frequency of 3 Hz was set as the known reference frequency, and a random waveform in the range of 3 ± 0.5 Hz was generated as the signal to be measured. The two waveforms were superimposed, and the envelope was obtained through Hilbert transform. Finally, the beat frequency was measured based on the peak and valley values of the envelope to obtain the frequency of the signal to be measured.

[0116] In the actual experiment, an LED light source that can blink at a frequency of 3 Hz was made to improve the measurement accuracy of low-frequency vibration. As Figure 12 shown, in the simulation results, δT is the time length corresponding to the beat frequency. According to the beat frequency data, By checking the workspace, it can be known that the frequency after adding random variations is exactly 3.3115 Hz, which provides strong support for improving the measurement accuracy of beat frequency measurement.

[0117] (5) Formal experiment

[0118] In this experiment, quantifiable regular vibrations will be measured. Regular vibrations are more likely to exhibit their characteristics in the instrument than irregular vibrations, and it is also easier to calibrate the accuracy of the device.

[0119] In the formal experiment, the coupled optical fiber will be used for signal collection to reduce the optical intensity loss caused by the inhomogeneity in the TPU optical fiber. Figure 13 The optical fiber devices to be used in the formal experiment are shown. Figure 13 In (a), it is the optical fiber coupling part coated with EVA glue. Figure 13 In (b), three optical fibers with a wire diameter of 1.4 mm are shown. Figure 13 In (c), it is a physical diagram of the optical fiber insertion position.

[0120] 5.1) Measurement of frequency

[0121] First, place the experimental device in a dark room, turn on the device light source, and exclude the interference of external light sources on the experiment. Connect the signal generator to the device to change the frequency of the excitation signal received by the experimental device. Fix the signal source as a 10V sine wave, start from 1Hz, increase by 1Hz every 15 seconds until it reaches 30Hz, repeat several times, and save the collected data as a picture file.

[0122] Use MATLAB software for data processing, quantify the information contained in the picture, and process the data collected in the experiment through three steps: comparison, fitting, and restoration.

[0123] 5.2) Measurement of amplitude

[0124] To further analyze the vibration characteristics in depth, the experiment will use an oscillation generator to generate three-dimensional vibrations in space. The experiment will then be carried out at fixed frequency points of 1Hz, 10Hz, and 19Hz in sequence. The signal source waveform is still a sine wave. At each fixed frequency, change the output signal intensity of the signal generator to change the amplitude of the vibration generator. The amplitude adjustment starts from the initial value of 100mV and increases by 100mV every 2 seconds until the maximum amplitude of 10V is reached.

[0125] (6) Data analysis

[0126] 6.1) Analysis of frequency

[0127] After calibration, the vibration of the light source can be transformed into the vibration of the detector by changing the reference frame. Taking the specific experimental conditions of a frequency of 4 Hz and an amplitude of 3 V as an example, the detector records and collects data on the variation of light intensity with time or length. See Figure 14 , after directly reading the signal from the signal source, using the light intensity-displacement conversion relationship, it is converted into a relative light intensity curve (blue in the figure), and compared with the data collected by the detector (scatter plot in the figure).

[0128] After the signal output by the signal source is directly received, through a preset light intensity-displacement conversion model, the signal is converted into a corresponding displacement change curve, which is represented by a blue line in the figure. The variation of the relative light intensity of the light source collected by the detector under the same experimental conditions is presented in the form of a scatter plot, as Figure 15 shown. See Figure 16 , through Fourier frequency domain analysis, this can be shown more intuitively; Figure 16 in (a), (b), and (c) are the Fourier spectra of the experimental results when the signal source frequency is 19 Hz, Figure 16 in (d) is the Fourier spectrum of the experimental results when the signal source frequency is 9 Hz. The first peak in the amplitude-frequency curve is the second-order fundamental frequency.

[0129] By Figures 14 - 16 it can be found that the frequencies of the curves are almost the same, but the amplitudes are different. After analyzing the experimental setup, it is found that the object to which the light source is attached itself also has different vibration modes, which will affect the vibration mode of the light source and thus the data collected by the detector.

[0130] The type A uncertainty in frequency measurement mainly comes from the information loss during computer information processing and the interference of other light sources. The expression is as follows:

[0131]

[0132] where N is the number of sampled segments, f i is the frequency obtained from each segment of data, in Hz, f0 is the signal frequency of the signal source, in Hz. The type B uncertainty comes from the information loss brought by each measuring device, including noise points, signal source signal errors, etc. Here, the uncertainty of the digital camera is taken:

[0133]

[0134] As the type B uncertainty, compared with the type A uncertainty, the type B uncertainty is small enough to be ignored during synthesis.

[0135] The uncertainty of the frequency is shown in Tables 3, 4, and 5 as follows:

[0136] Table 3: When f0=19Hz, there are six sets of data

[0137]

[0138] Average value of frequency

[0139] Frequency uncertainty

[0140] The final result

[0141] Table 4: When f0=10Hz, there are six sets of data

[0142]

[0143] Average value of frequency

[0144] Frequency uncertainty

[0145] The final result f0＝10Hz =10.1±0.1(Hz)

[0146] Table 5: When f0=5Hz, there are six sets of data

[0147]

[0148] Average value of frequency

[0149] Frequency uncertainty

[0150] The final result

[0151] 6.2) Amplitude Analysis

[0152] The three light sources of different colors vibrate at roughly the same fundamental frequency, and the amplitude depends on the distance of the detection point from the light source and the actual vibration amplitude. This section analyzes the vibration amplitude in the direction of each basis vector.

[0153] The curve obtained by Fourier analysis and removing interference items such as noise is subtracted from the actual vibration curves in all directions to obtain the residual graph, such as Figure 17 As shown in the figure, it can be found from the residual graph that the measured curve almost coincides with the true motion curve, proving that the experimental results are ideal. Figure 17 (a), (b), and (c) are the amplitude residual graphs of green light, blue light, and red light, respectively.

[0154] 1) Amplitude uncertainty

[0155] Similar to the calculation of frequency uncertainty, the type-A uncertainty of amplitude requires the relative light intensity I in N samplings i and the average value

[0156]

[0157] In the case of small amplitudes, the light intensity and length are approximately in a linear function relationship. That is, at a given frequency and amplitude, there is a linear trend between the change in light intensity and the corresponding change in length. Using the calibration results in the preliminary experiment, the converted uncertainty and the average amplitude. The amplitudes in three directions are as follows:

[0158] A x,10V = 0.931 ± 0.003 (mm)

[0159] A y,10V = 0.928 ± 0.002 (mm)

[0160] A z,10V = 0.955 ± 0.001 (mm)

[0161] 6.3) Restoration of spatial vibration

[0162] By synthesizing the data collected by three detectors (i.e., three optical fibers) respectively, the vibration curve of the light source in space can be obtained, and then the displacement of the vibrating object in space can be obtained.

[0163] Figure 18 It is the spatial synthesis curve of vibration at 10V in the vertical direction. The vibration frequency of the left figure is 6Hz, and the vibration frequency of the right figure is 8Hz. The amplitudes in other directions are the same.

[0164] Since it is difficult to record the actual vibration curve, here three high-speed cameras with mutually perpendicular viewing angles are used for high-speed photography simultaneously, and combined with the analysis results of the tracker software, and then synthesized to obtain an approximate actual vibration trajectory.

[0165] The restoration result of the data has a small difference from the actual motion curve, is relatively consistent with the real light source motion curve, and compared with the X-axis and Y-axis, there is a slight deviation between the restoration result in the Z-axis direction and the motion curve captured by Tracker. This is because after adding the receiver, the camera cannot be taken as close to the RGB light array as in the amplitude calibration in section 4.2), which makes it difficult to distinguish the small-amplitude vibration in the Z-axis, so the situation as Figure 19 shown occurs.

[0166] In some embodiments, since mechanical vibrations have distinct vibration modes, accurate vibration modes can be identified through machine learning vibration classification techniques. The application of this technology can not only improve the accuracy of vibration analysis but also significantly enhance the response speed and efficiency of the monitoring system. By deeply analyzing vibration data, the machine learning model can learn the characteristics of various vibration modes and apply them to actual vibration signal classification.

[0167] Machine learning models, especially deep learning models such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), can process complex vibration signals and extract key features from them. These features are then used to train the model to enable it to identify and distinguish different vibration patterns. In this way, the machine learning model can provide accurate information about the vibration state of the object to be monitored, enabling timely maintenance measures to be taken to avoid potential failures.

[0168] The experimental team built a simple machine learning model using a pre-trained LSTM network in Matlab. The model includes an input layer, an LSTM layer, a Dropout layer, a fully connected layer, a Softmax layer, and a classification output layer. The input layer receives sequence data with 12 feature dimensions, which are composed of 12 decoupled vibration signals (time series). The LSTM layer, with its unique memory cells and gating mechanism, can effectively capture the temporal dependencies in the sequence data, which is crucial for identifying complex patterns in vibration signals.

[0169] After the LSTM layer, the Dropout layer reduces the risk of overfitting in the model by randomly dropping out some neurons, improving the generalization ability of the model; the fully connected layer maps the high-dimensional output of the LSTM layer to a smaller dimensional space; the Softmax layer converts the output of the fully connected layer into a probability distribution, enabling the model to output the prediction probability for each class.

[0170] Finally, the classification output layer uses the cross-entropy loss function to measure the difference between the model prediction and the true label, and optimizes the network parameters through the backpropagation algorithm. The design of the entire model aims to achieve efficient classification of vibration signals to quickly identify abnormal vibration patterns in practical applications, thereby taking preventive maintenance measures.

[0171] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0172] The above embodiments are only for illustrating the technical concept and characteristics of the present invention, and the purpose is to enable those of ordinary skill in the art to understand the content of the present invention and implement it accordingly, and it should not be used to limit the protection scope of the present invention. All equivalent changes or modifications made according to the essence of the content of the present invention should be covered within the protection scope of the present invention.

Claims

1. An omnidirectional vibration monitoring system based on RGB images and fiber optic sensing, characterized in that Including: An RGB light array, which is set on the object to be measured; A receiver, with a fixed position and surrounding the RGB light array. When the RGB light array moves with the object to be measured, the light field in the receiver will change; Three optical fibers, which are respectively fixed at three directional positions of the receiver and are used to collect the light field information of the RGB three channels; A coupler, which is used to couple the information of the three optical fibers and is connected to a camera through one optical fiber; A camera, which is used to collect RGB images; A host computer, which is used to obtain the vibration information of the object to be measured according to the obtained RGB images.

2. The all-direction vibration monitoring system based on RGB images and fiber optic sensing according to claim 1, characterized in that, The three perspectives of the three optical fibers are perpendicular to each other. Each of the RGB three channels corresponds to one degree of freedom. Light field calibration is required before monitoring the object to be measured.

3. The omnidirectional vibration monitoring system based on RGB images and fiber optic sensing according to claim 1, characterized in that, The obtaining the vibration information of the object to be measured according to the obtained RGB images includes: By performing inverse solution on the RGB images, the light field information of the three channels is obtained; The RGB three color channels are respectively processed to obtain the brightness curves of each channel changing with time, and these curves reflect the change of light intensity during the vibration process; First, the pixel values of each channel are averaged to obtain the average brightness value of the channel; subsequently, by calculating the brightness change between consecutive frames, the light intensity fluctuation during the vibration process is traced; By analyzing the processed data, the characteristics of the vibration event are identified, and the vibration information of the object to be measured is obtained according to the identified characteristics; among them, the identified characteristics include the frequency, amplitude and possible vibration mode of the vibration.

4. The omnidirectional vibration monitoring system based on RGB images and fiber optic sensing according to claim 1, wherein The host computer is specifically used for: Calling the camera through Matlab programming to collect the video of the RGB color change caused by the vibration signal; then, extracting frames from the video and converting them into three-dimensional matrices, and respectively extracting the data of the RGB three channels; Applying a filter to smooth the data so as to clearly observe the periodicity and amplitude of the signal when plotting; finally, analyzing and visually displaying the processed data to verify the accuracy of the experimental results.

5. The omnidirectional vibration monitoring system based on RGB images and fiber optic sensing according to claim 4, characterized in that, The analyzing and visually displaying the processed data includes: Using Fourier analysis to calculate the frequency, and solving the three-dimensional vibration information of the object to be measured with the frequency movement data in three directions.

6. The omnidirectional vibration monitoring system based on RGB images and fiber optic sensing according to claim 4, wherein, If a USB-connected camera is used, according to the experimental device design, ensure that the optical fiber reading area matches the field of view angle of the camera, and calculate the average brightness in this area; then, perform normalization processing on the brightness data to eliminate the influence of illumination.

Images