Non-contact micro-vibration measurement and fault diagnosis method based on dynamic visual signal alignment

Abstract

A non-contact micro-vibration measurement and fault diagnosis method based on dynamic visual signal alignment comprises the steps that firstly, a dynamic visual sensor-event camera is used for collecting event data of vibration of a rotating machine in a non-contact mode, and an event data characterization method is used for converting event streams into event frame sequences which are equal in interval, not overlapped and continuous; then micro-vibration information of each pixel point is extracted from the event frame sequence through a deep neural network, the difference of vibration signals of adjacent pixel points is reduced through a signal alignment method, and contact sensing data is used as a reference for optimizing model performance through a feature clustering method; and finally, performing intelligent fault diagnosis by using the obtained micro-vibration data. According to the invention, the applicability of the non-contact sensor in the fields of vibration monitoring and fault diagnosis of rotating machinery is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of mechanical fault monitoring and diagnosis, and specifically relates to a non-contact micro-vibration measurement and fault diagnosis method based on dynamic visual signal alignment. Background Art

[0002] In modern industry, rotating machinery plays a key role in tasks such as manufacturing and transportation. The stable and precise operation of such machinery is essential to ensure production quality, efficiency and safety. However, the complex working environment significantly affects the performance of the machine, and in severe cases it can lead to equipment failure or even downtime. Therefore, it is very important to monitor the status of rotating machinery in a timely and effective manner.

[0003] At present, the condition monitoring of rotating machinery adopts various types of signals, including vibration, temperature, pressure, current, sound signals, etc. Among them, vibration signals are particularly important and effective for machine condition monitoring. Since it is highly sensitive to slight changes and faults in mechanical structure, vibration signals can well reveal the operating characteristics of rotating machinery.

[0004] Many studies use contact sensors, such as displacement, velocity and acceleration sensors. Although these sensors have high sensitivity and a wide frequency response range, they also have certain limitations: 1) Long-term installation of contact sensors will cause wear on the equipment surface and are not applicable in many industrial scenarios; 2) Rotating machinery is usually in extreme working environments such as high temperature and high pressure, which places high demands on contact sensors; 3) The signals collected by contact sensors are usually concentrated at a single measurement point. In order to reflect the large range of the machinery monitoring area, multiple sensors are required. At the same time, the installation position also needs to be appropriately selected.

[0005] In order to overcome the limitations of contact sensors, a new type of non-contact dynamic visual sensor, the event camera, has begun to be used in the field of monitoring and diagnosis. Standard cameras generate image frames containing the brightness information of all pixels in the field of view at a constant sampling frequency. In contrast, event cameras can capture relevant information from pixels whose brightness changes exceed a specified threshold and output an asynchronous event stream. This signal acquisition method gives event cameras unique dynamic characteristics advantages, such as high temporal resolution, wide dynamic range, and low power consumption, making them very suitable for capturing dynamic visual signals. Therefore, event cameras have great application potential in industrial fields such as edge tracking, autonomous driving, and high-speed positioning.

[0006] Although event cameras have many advantages, they also have challenges in micro-vibration measurement and fault diagnosis applications. Event data mainly captures the pixel position and corresponding time of vibration, but lacks actual detailed feature information. Therefore, event data cannot well reflect the precise equipment vibration; in addition, events occur at random times and locations. Therefore, converting event data into analyzable practical micro-vibration signals and applying these estimates to fault diagnosis is a complex and difficult task. Summary of the invention

[0007] In order to overcome the defects of the above-mentioned prior art, the purpose of the present invention is to provide a non-contact micro-vibration measurement and fault diagnosis method based on dynamic visual signal alignment, which uses an event camera to collect event data of rotating machinery vibration and extracts time-domain micro-vibration signals of each pixel point therefrom, and then uses a deep neural network to realize the fault classification of rotating machinery after signal alignment and feature clustering of the extracted signals, thereby improving the applicability of non-contact sensors in the field of vibration monitoring and fault diagnosis of rotating machinery.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is:

[0009] A non-contact micro-vibration measurement and fault diagnosis method based on dynamic visual signal alignment first uses a dynamic visual sensor - an event camera to non-contactly collect event data of rotating machinery vibration, and uses an event data characterization method to convert the event stream into an equally spaced, non-overlapping and continuous event frame sequence; then, a deep neural network is used to extract the micro-vibration information of each pixel in the event frame sequence, and the difference in vibration signals of adjacent pixels is narrowed by a signal alignment method. The contact sensing data is used as a reference for optimizing model performance through a feature clustering method; finally, the obtained micro-vibration data is used for intelligent fault diagnosis.

[0010] A non-contact micro-vibration measurement and fault diagnosis method based on dynamic visual signal alignment comprises the following steps:

[0011] Step 1, event data collection: using an event camera to collect asynchronous event streams of rotating machinery vibration;

[0012] Step 2, event data characterization: First, the event stream is reconstructed into an event vibration sequence, and then used as a sample for extracting vibration signals; the collected event stream data is reconstructed using a two-dimensional representation method for event data, and a segment of event stream data is divided into equal-length, non-overlapping, and continuous sub-event stream data; in the sub-event stream data, for each pixel, the polarity of each event is superimposed, and the sub-event stream is compressed into a two-dimensional event frame; and the entire event stream is reconstructed into an event frame sequence, specifically represented as where p irepresents the polarity of the event, (x, y) represents the location of the event, Ω represents the two-dimensional field of view of the event frame, and n d Indicates the specified time period t d The total number of events at this pixel;

[0013] The data of the contact sensor is used to calibrate the vibration signal extracted from the event frame sequence and perform feature difference analysis. First, the data of the contact sensor corresponding to each operating state of the rotating machinery is collected, and then the relevant features are extracted as correction labels. The i-th event data sample is represented as s i =(r i ,h i ,j i ), where r i =[r i1 , r i2 ,…,r in ] represents an event frame sequence, n represents the number of consecutive event frames contained in the event frame sequence; h i Indicates the machine health status label; j i A calibration label representing the acceleration signal characteristics under the corresponding state;

[0014] Step 3, establish a model of a micro-vibration monitoring and fault diagnosis network framework based on event signals, which consists of two parts: a vibration signal extractor and a fault pattern recognizer; the vibration signal extractor extracts the time domain vibration signal of each pixel from the input event frame sequence, and then sends it to the fault pattern recognizer to identify its fault mode;

[0015] Step 4, vibration signal alignment: Assume that the object monitored by the event camera is a rigid body, each pixel vibrates synchronously with the adjacent pixels, and the time domain vibration signals of these pixels are similar. This assumption is used as the basis for the deviation correction of the time domain vibration signals of adjacent pixels.

[0016]

[0017] Among them, L d represents the corresponding loss; (x, y) represents the coordinates of the collected pixels; S (x,y) Represents the time domain vibration signal corresponding to the selected pixel point; Represents the time domain vibration signal of adjacent pixel points; a and b represent even numbers of adjacent points in the X and Y directions respectively;

[0018] Step 5, vibration feature clustering: For the samples in the training set, calculate the time domain features of their vibration signals and cluster them according to the working conditions. The cluster center is defined as the time domain features calculated based on the data of the contact sensor. The intra-class loss of each class is calculated based on the cluster center, and the losses of all classes are aggregated to form the final cluster loss.

[0019]

[0020] Q j =[q j1 ,q j2 ,…,q jr ]

[0021] q jr =[A,B,C]

[0022]

[0023]

[0024] Where, L e represents the total clustering loss, w i represents the intra-class loss of each class, that is, the Euclidean distance between the feature vector of the samples contained in each class and the central feature vector; K represents the total number of classes; Q acc represents the central eigenvector; N e Indicates the total number of samples in the class; Q j Represents the feature vector, which contains the time domain feature q of each vibration signal jr ; Wherein, A, B and C represent the peak index, margin index and skew index in the time domain characteristics respectively, and these indexes are dimensionless parameters;

[0025] Step 6, model training: initialize network parameters, input training samples into the model to calculate classification loss, alignment loss and clustering loss, which are used to optimize network parameters; after each epoch training is completed, discard the samples of that epoch until the model converges; finally, input unlabeled test data into the model to evaluate the test performance.

[0026] Step 1 For a single pixel, the event camera records the brightness of the pixel at each moment with a resolution of microseconds and compares it with the brightness at the previous moment:

[0027] ±C=logI(x,y,t)-logI(x,y,t-Δt)

[0028] Where C represents contrast sensitivity, i.e., the threshold; I(x,y,t) represents the brightness of the pixel at coordinate (x,y) in the two-dimensional resolution region at time t; Δt represents the time interval; when the logarithmic change of brightness exceeds the threshold, the event camera will mark and output the event related to the pixel; where the change exceeds the positive threshold, a positive event is output; where the change exceeds the negative threshold, a negative event is output; the format of the event data is a four-element vector e = [t,x,y,p], where t represents the timestamp; x and y represent the horizontal and vertical coordinates of the pixel in the resolution region, respectively; p represents the polarity of the event: p = +1 represents that the logarithmic change of brightness exceeds the positive threshold, and p = -1 represents that the logarithmic change of brightness exceeds the negative threshold; the event stream represents the set of all events captured by the event camera over a period of time, expressed as Among them, e i represents the i-th event, n e For time period t e The total number of events recorded in the .

[0029] Step 3 introduces Gabor transform to extract texture features of different scales and directions in the image, so as to obtain more accurate local motion information; Gabor transform is based on Gabor function, which is the product of complex sine wave and Gaussian function, specifically expressed as:

[0030]

[0031] Where g(x,y,λ,θ,ψ,σ,γ) represents the Gabor filter; (x,y) is the two-dimensional coordinate of the selected pixel; (x′,y′) is the rotated two-dimensional coordinate; λ is the wavelength of the sinusoidal plane wave, θ is the direction of the parallel stripes of the filter; ψ is the phase shift, σ is the standard deviation of the Gaussian kernel, and γ is the spatial aspect ratio;

[0032] Assuming that the intensity of the image frame at the moment is E(x, y, t), the image frame is convolved with the Gabor filter to obtain the frequency domain information of the image frame:

[0033]

[0034] Where e(x,y,t) represents the frequency domain information of the image frame;

[0035] In the time period Δt, if there is local motion in the image at a certain location (x, y), the intensity of the image frame before and after the motion is E(x, y, t) and E(x+Δx, y+Δy, t+Δt), respectively. Assuming that the parallel stripe direction of the Gabor filter is 0°, the frequency domain information before and after the motion is expressed in double integral form as follows:

[0036]

[0037] By analyzing the above two equations, we can get the phase difference The relationship between the horizontal displacement Δx is:

[0038]

[0039] A hybrid method of Gabor transform and deep neural network is adopted. The convolution kernel generated by Gabor transform replaces the convolution kernel of the first layer of deep neural network to extract the underlying features of edge and texture of the image. For each event frame sequence sample of the feature extractor, the first layer of the network adopts a Gabor filter with fixed parameters to perform hierarchical convolution on each event frame in the sample. Each event frame corresponds to a Gabor filter, so that the number of channels and event frame sequence of the sample after convolution are the same as those of the sample before convolution. Then the texture features are input into the deep neural network for more detailed feature extraction. The deep neural network contains two residual blocks, each of which adopts the convolution kernel k s The samples are layered convolved and activated using the LeakyReLU function. Next, the texture features are superimposed on the features output by the deep neural network to achieve subtle adjustments to the texture features, ultimately resulting in a set of frame sequences, each of which contains the vibration features of each pixel.

[0040] Based on the extracted vibration features, the time domain vibration signals of all pixels are obtained, and then the vibration estimation of multiple pixel points of the rotating machinery is realized; first, a pixel point is selected, and this pixel point needs to be a pixel point in the rotating machinery area in the frame; secondly, the values ​​of the pixel points in each frame of the frame sequence are extracted to form a time series, and the first point is determined as the reference point, and the other points are calibration points; then, the difference between each calibration point and the reference point is calculated in turn to realize the relative displacement of the pixel points at these two moments, and finally the time domain vibration signal of the pixel point is obtained through iteration;

[0041] The obtained time-domain vibration signal is input into the fault mode recognizer, which consists of a one-dimensional deep neural network. The network contains two residual blocks. The convolution layer in each residual block uses a convolution kernel of length 9; the first residual block increases the dimension of the signal to 20 dimensions to extract its high-dimensional features and reduces its length by half; the second residual block further reduces its length by half; after flattening, the sample features are connected to two fully connected layers, each with 64 neurons, and the number of neurons represents the number of related health conditions. Finally, the softmax function is used for classification.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] This paper proposes a non-contact micro-vibration measurement and fault diagnosis method based on dynamic visual signal alignment. By analyzing the visual data captured by the event camera, the time domain vibration signal is extracted from each pixel in the event stream, and then the extracted time domain signal is applied to the recognition of the machine health status, realizing non-contact, global vibration monitoring and fault diagnosis. In order to enhance the robustness of the model, the signal alignment method is used to minimize the difference in vibration signals between adjacent pixels. In addition, a feature clustering method is introduced, and the contact sensing data is used as a reference for optimizing the model performance. Experimental results show that the method of the present invention effectively extracts vibration features from dynamic visual data under various working conditions and accurately identifies the type of machine faults, and has a bright future in the field of industrial applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of a flow chart of an embodiment of the present invention.

[0045] Figure 2 It is a schematic diagram of a vibration frequency domain signal converted from an event signal according to an embodiment of the present invention.

[0046] Figure 3 1 is a confusion matrix diagram of the embodiment of the present invention and the comparative method.

[0047] Figure 4 TSNE visualization diagram of the embodiment of the present invention and the comparative method. DETAILED DESCRIPTION

[0048] The present invention is further described in detail below in conjunction with embodiments and drawings.

[0049] Reference Figure 1 , a non-contact micro-vibration measurement and fault diagnosis method based on dynamic visual signal alignment, comprising the following steps:

[0050] Step 1, event data acquisition: Use the event camera to collect the asynchronous event stream of rotating machinery vibration; for a single pixel, the event camera records the brightness of the pixel at each moment with a resolution of microseconds and compares it with the brightness of the previous moment:

[0051] ±C=logI(x,y,t)-logI(x,y,t-Δt)

[0052] Where C represents contrast sensitivity, i.e., the threshold; I(x,y,t) represents the brightness of the pixel at coordinate (x,y) in the two-dimensional resolution region at time t; Δt represents the time interval; when the logarithmic change of brightness exceeds the threshold, the event camera will mark and output the event related to the pixel; where the change exceeds the positive threshold, a positive event is output; where the change exceeds the negative threshold, a negative event is output; the format of the event data is a four-element vector e = [t,x,y,p], where t represents the timestamp; x and y represent the horizontal and vertical coordinates of the pixel in the resolution region, respectively; p represents the polarity of the event: p = +1 represents that the logarithmic change of brightness exceeds the positive threshold, and p = -1 represents that the logarithmic change of brightness exceeds the negative threshold; for the event stream, it represents the set of all events captured by the event camera over a period of time, expressed as Among them, e i represents the i-th event, n e For time period t e The total number of events recorded in

[0053] Step 2, event data characterization: It is difficult to extract vibration information directly from the event stream, because these data do not represent the actual vibration signal of the rotating machinery, and the event data cannot be directly divided into samples. To address this problem, the event stream is first reconstructed into an event vibration sequence, and then used as a sample for extracting the vibration signal; specifically, the collected event stream data is reconstructed using a two-dimensional representation method for event data, and a segment of event stream data is divided into equal-length, non-overlapping, and continuous sub-event stream data; in the sub-event stream data, for each pixel, the polarity of each event is superimposed, so the sub-event stream is compressed into a two-dimensional event frame; and the entire event stream is reconstructed into an event frame sequence, specifically represented as where p i represents the polarity of the event, (x, y) represents the location of the event, Ω represents the two-dimensional field of view of the event frame, and n d Indicates the specified time period t d The total number of events at the pixel within the event stream; this representation method can effectively integrate the event stream in the time and space dimensions, making the characteristics of the event stream more prominent. At the same time, it also converts the complex event stream data into typical frame data, which is convenient for the subsequent extraction of micro-vibration signals from the event frame;

[0054] In machine condition monitoring, contact vibration sensors such as accelerometers usually have accurate measurements and stable performance. Therefore, the data of contact sensors can be used to calibrate the vibration signals extracted from the event frame sequence. Since the two signals do not correspond one to one, feature difference analysis is required. Taking the acceleration data of the accelerometer as an example, the acceleration data corresponding to each operating state of the machine is first collected, and then the relevant features are extracted as correction labels. Therefore, the i-th event data sample can be expressed as s i =(ri ,h i ,j i ), where r i =[r i1 , r i2 ,…,r in ] represents an event frame sequence, n represents the number of consecutive event frames contained in the event frame sequence; h i Indicates the machine health status label. i A calibration label representing the acceleration signal characteristics under the corresponding state;

[0055] Step 3, establish a model of a micro-vibration monitoring and fault diagnosis network framework based on event signals, which mainly consists of two parts: a vibration signal extractor and a fault pattern recognizer; the vibration signal extractor extracts the time domain vibration signal of each pixel from the input event frame sequence, and then sends it to the fault pattern recognizer to identify its fault mode;

[0056] In order to extract the local motion phase from the event frame sequence, the Gabor transform is introduced. The Gabor transform is a short-time windowed Fourier transform with good time domain localization and frequency domain bandpass characteristics. It can well extract texture features of different scales and directions in the image, thereby obtaining more accurate local motion information. The Gabor transform has a wide range of applications in image processing, pattern recognition and other fields. The Gabor transform is based on the Gabor function, which is the product of a complex sine wave and a Gaussian function, specifically expressed as,

[0057]

[0058] Where g(x,y,λ,θ,ψ,σ,γ) represents the Gabor filter; (x,y) is the two-dimensional coordinate of the selected pixel; (x′,y′) is the rotated two-dimensional coordinate; λ is the wavelength of the sinusoidal plane wave, θ is the direction of the parallel stripes of the filter; ψ is the phase shift, σ is the standard deviation of the Gaussian kernel, and γ is the spatial aspect ratio.

[0059] Assuming that the intensity of the image frame at the moment is E(x, y, t), the image frame is convolved with the Gabor filter to obtain the frequency domain information of the image frame:

[0060]

[0061] Where e(x,y,t) represents the frequency domain information of the image frame;

[0062] In the time period Δt, if there is local motion in the image at a certain location (x, y), the intensity of the image frame before and after the motion is E(x, y, t) and E(x+Δx, y+Δy, t+Δt), respectively. Assuming that the parallel stripe direction of the Gabor filter is 0°, the frequency domain information before and after the motion is expressed in double integral form as follows:

[0063]

[0064] By analyzing the above two equations, we can get the phase difference The relationship between the horizontal displacement Δx is:

[0065]

[0066] In order to improve the performance of Gabor transform, a hybrid method of Gabor transform and deep neural network is adopted. The convolution kernel generated by Gabor transform replaces the convolution kernel of the first layer of deep neural network, which can more effectively extract the underlying features of the image such as edges and textures, thereby improving the perception ability. Specifically, for each event frame sequence sample of the feature extractor, the first layer of the network uses a Gabor filter with fixed parameters to perform hierarchical convolution on each event frame in the sample. Each event frame corresponds to a Gabor filter, so that the number of channels and event frame order of the sample after convolution are the same as those of the sample before convolution. Then the texture features are input into the deep neural network for more detailed feature extraction. The deep neural network contains two residual blocks, each of which uses the convolution kernel k s The samples are layered convolved and activated using the LeakyReLU function. Next, the texture features are superimposed on the features output by the deep neural network to achieve subtle adjustments to the texture features. Finally, a set of frame sequences can be obtained, in which each frame contains the vibration features of each pixel.

[0067] Based on the extracted vibration features, the time domain vibration signals of all pixels can be obtained, and then the vibration estimation of multiple pixel points of the rotating machinery can be realized; first, a suitable pixel point is selected, and this pixel point needs to be a pixel point in the rotating machinery area in the frame; secondly, the values ​​of the pixel points in each frame of the frame sequence are extracted to form a time series, and the first point is determined as the reference point, and the other points are calibration points; then, the difference between each calibration point and the reference point is calculated in turn, and then the relative displacement of the pixel points at these two moments is realized. Through iteration, the time domain vibration signal of the pixel point is finally obtained;

[0068] The obtained time-domain vibration signal is input into the fault mode recognizer, which is composed of a one-dimensional deep neural network. The network contains two residual blocks. The convolution layer in each residual block uses a convolution kernel of length 9; the first residual block increases the dimension of the signal to 20 dimensions to extract its high-dimensional features and reduces its length by half; the second residual block further reduces its length by half; after flattening, the sample features are connected to two fully connected layers, each with 64 neurons, and the number of neurons represents the number of related health conditions. Finally, the softmax function is used for classification;

[0069] Step 4, vibration signal alignment: Assume that the object monitored by the event camera is a rigid body, each pixel vibrates synchronously with the adjacent pixels, and the time domain vibration signals of these pixels are similar. This assumption is used as the basis for the deviation correction of the time domain vibration signals of adjacent pixels.

[0070]

[0071] Among them, L d represents the corresponding loss; (x, y) represents the coordinates of the collected pixels; S (x,y) Represents the time domain vibration signal corresponding to the selected pixel point; Represents the time domain vibration signal of adjacent pixel points; a and b represent even numbers of adjacent points in the X and Y directions respectively;

[0072] Step 5, vibration feature clustering: For different samples collected under the same working condition and healthy state, the time domain signals obtained through the network will be different, and it is very difficult to calculate the error of each point, but the time domain features of these signals have certain similarities, so the intra-class loss can be reduced by clustering these features; specifically, for the samples in the training set, the time domain features of their vibration signals are calculated and clustered according to the working conditions. The cluster center is defined as the time domain features calculated based on the accelerometer acquisition signal of this category. The intra-class loss of each class is calculated based on the cluster center, and the losses of all classes are aggregated to form the final clustering loss.

[0073]

[0074] Q j =[q j1 ,q j2 ,…,q jr ]

[0075] q jr =[A,B,C]

[0076]

[0077] Where, L e represents the total clustering loss, w irepresents the intra-class loss of each class, that is, the Euclidean distance between the feature vector of the samples contained in each class and the central feature vector; K represents the total number of classes; Q acc represents the central eigenvector; N e Indicates the total number of samples in the class; Q j Represents the feature vector, which contains the time domain feature q of each vibration signal jr ; Where A, B and C represent the peak index, margin index and skew index in the time domain characteristics respectively. These indexes are dimensionless parameters;

[0078] Step 6, model training: initialize network parameters, input training samples into the model to calculate classification loss, alignment loss and clustering loss, which are used to optimize network parameters; after each epoch training is completed, discard the samples of that epoch until the model converges; finally, input unlabeled test data into the model to evaluate the test performance.

[0079] The embodiment takes a rolling bearing in a rotating machine as an example, and verifies the effectiveness of the method of the present invention based on rolling bearing experimental data.

[0080] The test bench uses a motor to drive the shaft through a coupling, and an ER-16K rolling bearing is installed on the shaft. The parameters are shown in Table 1; the event camera is placed in front of the bearing to collect event data of bearing vibration; this experiment uses an event camera of model Prophesee 3.1, and the parameters are shown in Table 2; the acceleration sensor is placed above the bearing to synchronously collect acceleration vibration signals for comparative analysis, and the sampling rate of the acceleration sensor is 12.8kHz; the experiment considers four typical rolling bearing health states, namely health, outer ring fault, inner ring fault and rolling element fault; at the same time, experimental data are collected at five different speeds, and the specific experimental conditions are shown in Table 3; the model hyperparameters of this embodiment are shown in Table 4;

[0081] Table 1 Rolling bearing parameters

[0082]

[0083] Table 2 Event camera parameters

[0084]

[0085]

[0086] Table 3 Experimental conditions

[0087]

[0088] Table 4 Model hyperparameter settings

[0089]

[0090] Various methods are evaluated to illustrate the effectiveness and superiority of the proposed fault diagnosis approach within the cross-modal learning framework:

[0091] 1) NoA method (no alignment): In order to evaluate the advantages of the proposed feature alignment method, the NoA method was implemented. In this case, the feature alignment method was not considered, thereby ignoring the alignment loss of vibration signals at different pixels of the same sample during the model optimization process. All other parameters remained consistent with the proposed method.

[0092] 2) NoC method (no clustering): The performance enhancement of the feature clustering method is demonstrated by the NoC method, in which the feature clustering method is not used during model optimization, only the feature alignment method is used, while all other parameters are generally kept the same as those of the proposed method.

[0093] 3) NoAC methods (No Alignment and Clustering): Deep neural network methods represent the standard approach for fault diagnosis modeling, following the supervised learning paradigm, in which only the cross-entropy loss function for dynamic visual data is considered in model optimization, while feature alignment and clustering methods are excluded from the proposed network architecture.

[0094] 4) NoGF method (without Gabor filter): In order to evaluate the effectiveness of Gabor filters for event frame vibration feature extraction, the Gabor filters were removed from the compared methods and the first layer of the feature extractor was replaced with a standard convolution kernel that was randomly initialized and continuously optimized throughout the training process instead of using a Gabor filter with fixed parameters.

[0095] The time domain micro-vibration signal extracted by the embodiment is converted into a frequency domain signal through Fourier transform, and its spectrum is as follows: Figure 2 As shown, for different rotation speeds, the rotation frequency of the device and its harmonics can be clearly found in the spectrum, indicating that the embodiment method can effectively extract the micro-vibration signal of each pixel from the event data, laying the foundation for the subsequent identification of different fault modes.

[0096] Table 5 Classification accuracy of different methods

[0097]

[0098] Table 5 lists the overall experimental results of various fault diagnosis tasks using different methods. It can be found that the embodiment method achieves a test accuracy of more than 98%. In contrast, the alternative event data processing methods (NoA, NoC, and NoAC) show competitive performance, and the test accuracy is usually above 90%. However, compared with the embodiment method, the accuracy of these methods is slightly lower, indicating that time domain signal alignment and feature clustering help improve model performance. In contrast, the classification accuracy of NoGF is relatively low, less than 50%, which shows that the basic deep neural network cannot effectively capture the real time domain vibration signal. The above results confirm the feasibility of fault diagnosis using vibration signals extracted from dynamic visual data.

[0099] In order to show the detailed test results of various machine failures, Figure 3 The confusion matrices of the embodiment method and the comparison method are shown. The comparison shows that without the Gabor filter, the model can only accurately identify rolling element faults at 30Hz and 40Hz, and cannot distinguish other fault modes and healthy states. In contrast, the recognition accuracy of the embodiment method for each fault mode reaches more than 98%, which exceeds the performance of other comparison methods in the corresponding category.

[0100] The T-SNE technique is used to visualize and evaluate the features learned by various methods, with a special focus on the data representation of the fully connected layer of the fault pattern recognizer. The results are shown in Figure 2. Figure 4 As shown in the visualization results generated by the NoGF method, it can be clearly seen that most samples are clustered in the same area with the smallest separation; although the NoAC method shows a high accuracy, its cross-category clustering effect is not ideal, and there is obvious overlap at the category boundaries. In contrast, the embodiment method shows a significantly improved clustering effect for samples representing different machine health states, with clearer category boundaries and enhanced separation, which proves the effectiveness and superiority of the embodiment method in identifying machine failure modes based on event data.

Claims

1. A non-contact micro-vibration measurement and fault diagnosis method based on dynamic visual signal alignment, characterized in that: First, a dynamic visual sensor, an event camera, is used to non-contactly collect event data of rotating machinery vibration, and an event data characterization method is used to convert the event stream into an equally spaced, non-overlapping and continuous event frame sequence. Then, a deep neural network is used to extract the micro-vibration information of each pixel in the event frame sequence, and the difference in vibration signals of adjacent pixels is narrowed by a signal alignment method. The contact sensing data is used as a reference for optimizing model performance by a feature clustering method. Finally, the obtained micro-vibration data is used for intelligent fault diagnosis.

2. A non-contact micro-vibration measurement and fault diagnosis method based on dynamic visual signal alignment, characterized in that: The following steps are involved: Step 1, event data collection: using an event camera to collect asynchronous event streams of rotating machinery vibration; Step 2, event data characterization: First, the event stream is reconstructed into an event vibration sequence, and then used as a sample for extracting vibration signals; the collected event stream data is reconstructed using a two-dimensional representation method for event data, and a segment of event stream data is divided into equal-length, non-overlapping, and continuous sub-event stream data; in the sub-event stream data, for each pixel, the polarity of each event is superimposed, and the sub-event stream is compressed into a two-dimensional event frame; and the entire event stream is reconstructed into an event frame sequence, specifically represented as where p i represents the polarity of the event, (x, y) represents the location of the event, Ω represents the two-dimensional field of view of the event frame, and n d Indicates the specified time period t d The total number of events at this pixel; The data of the contact sensor is used to calibrate the vibration signal extracted from the event frame sequence and perform feature difference analysis. First, the data of the contact sensor corresponding to each operating state of the rotating machinery is collected, and then the relevant features are extracted as correction labels. The i-th event data sample is represented as s i =(r i ,h i ,j i ), where r i =[r i1 , r i2 ,…,r in ] represents an event frame sequence, n represents the number of consecutive event frames contained in the event frame sequence; h i Indicates the machine health status label; j i A calibration label representing the acceleration signal characteristics under the corresponding state; Step 3, establish a model of a micro-vibration monitoring and fault diagnosis network framework based on event signals, which consists of two parts: a vibration signal extractor and a fault pattern recognizer; the vibration signal extractor extracts the time domain vibration signal of each pixel from the input event frame sequence, and then sends it to the fault pattern recognizer to identify its fault mode; Step 4, vibration signal alignment: Assume that the object monitored by the event camera is a rigid body, each pixel vibrates synchronously with the adjacent pixels, and the time domain vibration signals of these pixels are similar. This assumption is used as the basis for the deviation correction of the time domain vibration signals of adjacent pixels. Among them, L d represents the corresponding loss; (x, y) represents the coordinates of the collected pixels; S (x,y) Represents the time domain vibration signal corresponding to the selected pixel point; Represents the time domain vibration signal of adjacent pixel points; a and b represent even numbers of adjacent points in the X and Y directions respectively; Step 5, vibration feature clustering: For the samples in the training set, calculate the time domain features of their vibration signals and cluster them according to the working conditions. The cluster center is defined as the time domain features calculated based on the data of the contact sensor. The intra-class loss of each class is calculated based on the cluster center, and the losses of all classes are aggregated to form the final cluster loss. Q j =[q j1 ,q j2 ,…,q jr ] q jr =[A,B,C] Where, L e represents the total clustering loss, w i represents the intra-class loss of each class, that is, the Euclidean distance between the feature vector of the samples contained in each class and the central feature vector; K represents the total number of classes; Q acc represents the central eigenvector; N e Indicates the total number of samples in the class; Q j Represents the feature vector, which contains the time domain feature q of each vibration signal jr ; Wherein, A, B and C represent the peak index, margin index and skew index in the time domain characteristics respectively, and these indexes are dimensionless parameters; Step 6, model training: initialize network parameters, input training samples into the model to calculate classification loss, alignment loss and clustering loss, which are used to optimize network parameters; after each epoch training is completed, discard the samples of that epoch until the model converges; finally, input unlabeled test data into the model to evaluate the test performance.

3. The method according to claim 2, characterized in that: Step 1 For a single pixel, the event camera records the brightness of the pixel at each moment with a resolution of microseconds and compares it with the brightness at the previous moment: ±C=logI(x,y,t)-logI(x,y,t-Δt) Where C represents contrast sensitivity, i.e., the threshold; I(x,y,t) represents the brightness of the pixel at coordinate (x,y) in the two-dimensional resolution region at time t; Δt represents the time interval; when the logarithmic change of brightness exceeds the threshold, the event camera will mark and output the event related to the pixel; where the change exceeds the positive threshold, a positive event is output; where the change exceeds the negative threshold, a negative event is output; the format of the event data is a four-element vector e = [t,x,y,p], where t represents the timestamp; x and y represent the horizontal and vertical coordinates of the pixel in the resolution region, respectively; p represents the polarity of the event: p = +1 represents that the logarithmic change of brightness exceeds the positive threshold, and p = -1 represents that the logarithmic change of brightness exceeds the negative threshold; the event stream represents the set of all events captured by the event camera over a period of time, expressed as Among them, e i represents the i-th event, n e For time period t e The total number of events recorded in the .

4. The method according to claim 2, characterized in that: Step 3 introduces Gabor transform to extract texture features of different scales and directions in the image, so as to obtain more accurate local motion information; Gabor transform is based on Gabor function, which is the product of complex sine wave and Gaussian function, specifically expressed as: Where g(x,y,λ,θ,ψ,σ,γ) represents the Gabor filter; (x,y) is the two-dimensional coordinate of the selected pixel; (x′,y′) is the rotated two-dimensional coordinate; λ is the wavelength of the sinusoidal plane wave, θ is the direction of the parallel stripes of the filter; ψ is the phase shift, σ is the standard deviation of the Gaussian kernel, and γ is the spatial aspect ratio; Assuming that the intensity of the image frame at the moment is E(x, y, t), the image frame is convolved with the Gabor filter to obtain the frequency domain information of the image frame: Where e(x,y,t) represents the frequency domain information of the image frame; In the time period Δt, if there is local motion in the image at a certain location (x, y), the intensity of the image frame before and after the motion is E(x, y, t) and E(x+Δx, y+Δy, t+Δt), respectively. Assuming that the parallel stripe direction of the Gabor filter is 0°, the frequency domain information before and after the motion is expressed in double integral form as follows: By analyzing the above two equations, we can get the phase difference The relationship between the horizontal displacement Δx is: A hybrid method of Gabor transform and deep neural network is adopted. The convolution kernel generated by Gabor transform replaces the convolution kernel of the first layer of deep neural network to extract the underlying features of edge and texture of the image. For each event frame sequence sample of the feature extractor, the first layer of the network adopts a Gabor filter with fixed parameters to perform hierarchical convolution on each event frame in the sample. Each event frame corresponds to a Gabor filter, so that the number of channels and event frame sequence of the sample after convolution are the same as those of the sample before convolution. Then the texture features are input into the deep neural network for more detailed feature extraction. The deep neural network contains two residual blocks, each of which adopts the convolution kernel k s The samples are layered convolved and activated using the LeakyReLU function. Next, the texture features are superimposed on the features output by the deep neural network to achieve subtle adjustments to the texture features, ultimately resulting in a set of frame sequences, each of which contains the vibration features of each pixel. Based on the extracted vibration features, the time domain vibration signals of all pixels are obtained, and then the vibration estimation of multiple pixel points of the rotating machinery is realized; first, a pixel point is selected, and this pixel point needs to be a pixel point in the rotating machinery area in the frame; secondly, the values ​​of the pixel points in each frame of the frame sequence are extracted to form a time series, and the first point is determined as the reference point, and the other points are calibration points; then, the difference between each calibration point and the reference point is calculated in turn to realize the relative displacement of the pixel points at these two moments, and finally the time domain vibration signal of the pixel point is obtained through iteration; The obtained time-domain vibration signal is input into the fault mode recognizer, which is composed of a one-dimensional deep neural network. The network contains two residual blocks. The convolution layer in each residual block adopts a convolution kernel of length 9. The first residual block increases the dimension of the signal to 20 dimensions to extract its high-dimensional features and reduces its length by half. The second residual block further reduces its length by half. After flattening, the sample features are connected to two fully connected layers, each with 64 neurons. The number of neurons represents the number of related health conditions. Finally, the softmax function is used for classification.

Images