Centrifugal pump fault diagnosis method and system based on visual sensing technology

Abstract

The invention relates to a centrifugal pump fault diagnosis method and system based on a visual sensing technology. The system comprises a video stabilization module, a vibration amplification module, a vibration measurement module, a fault diagnosis module and a data processing and storage module. And the video stabilization module is used for processing a video by adopting an SIFT (Scale-Invariant Feature Transform) algorithm and an RANSAC (Random Sample Consensus) algorithm, so that the jitter of a camera is eliminated. And the vibration amplification module is used for realizing vibration visualization by using an improved N-PBVM (Non-Fourier Phasease-based Video Magnitation) algorithm (N-PBVM), and the vibration amplification module is used for realizing the vibration visualization by using the improved N-PBVM algorithm (Non-Fourier Phasease-based Video Magnitation, N-PBVM). The vibration measurement module calculates vibration displacement through an optical flow method related to an image phase. The fault diagnosis module identifies fault features based on frequency domain analysis, such as rotor imbalance, misalignment, and base looseness. And the data processing and storage module stores and analyzes the vibration data and predicts a fault trend by adopting a machine learning algorithm. According to the system, non-contact measurement and fault diagnosis of the centrifugal pump are achieved, and an efficient and reliable technical means is provided for maintenance of industrial equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and vibration measurement, and in particular to a centrifugal pump operation video stabilization and vibration measurement system and method based on visual sensing technology. Background Art

[0002] Centrifugal pumps are widely used in industrial and civil fields. For example, in industries such as petroleum, chemical, electricity, water supply and sewage treatment, centrifugal pumps play a key role as important conveying equipment. However, centrifugal pumps are often affected by various vibration problems during operation, resulting in unstable equipment operation, reduced service life, and even equipment failure and shutdown. Traditional vibration measurement methods mainly rely on contact sensors, such as accelerometers and displacement sensors, which need to be directly installed on the equipment, and have problems such as complex installation, difficult maintenance, and limited measurement range.

[0003] With the development of visual sensing technology, non-contact visual sensing vibration measurement methods have gradually become an emerging alternative. Visual sensors can remotely obtain the operating status of equipment and realize real-time monitoring and fault diagnosis of equipment by analyzing the vibration information in video images. However, in practical applications, due to the complexity of the camera installation environment and the inevitable jitter during operation, the video source is unstable, which seriously affects the accuracy of vibration measurement and fault diagnosis.

[0004] In the prior art, the phase-based vibration measurement (PBVM) algorithm has attracted widespread attention because it can amplify and visualize tiny vibrations through video processing technology. The PBVM algorithm accurately amplifies and visualizes the vibration of the structure by extracting phase information from the video frame. However, the traditional PBVM algorithm has the problems of high computational complexity and low processing efficiency in practical applications. Especially in the process of high-resolution video processing, the algorithm has a large computational load and it is difficult to achieve real-time processing. In addition, when faced with complex environmental noise, the traditional PBVM algorithm has weak anti-interference ability and is easily affected by noise, resulting in inaccurate vibration measurement results.

[0005] In view of the above problems, a centrifugal pump operation video stabilization and vibration measurement system based on visual sensing technology is proposed. The scale-invariant feature transform (SIFT) algorithm and random sampling consensus (RANSAC) algorithm are used to stabilize the video data to eliminate the video instability caused by camera shaking. At the same time, the optimized PBVM algorithm is used for vibration amplification to achieve accurate monitoring and vibration analysis of the centrifugal pump operation status, providing an efficient and reliable technical means for fault diagnosis and maintenance of industrial equipment. Summary of the invention

[0006] The present invention aims to solve the problems in the prior art that the vibration displacement amplitude of centrifugal pumps is small and difficult to observe and analyze intuitively, as well as the shortcomings of the traditional PBVM algorithm, which has high computational complexity, low processing efficiency, and weak anti-interference ability. An optimized PBVM algorithm and its application in centrifugal pump vibration visualization processing are proposed. Through a series of innovative optimization strategies, the algorithm's computational efficiency is significantly improved, while ensuring the accuracy and reliability of vibration measurement results.

[0007] The technical solution adopted by the present invention is a centrifugal pump fault diagnosis system based on visual sensing technology, comprising the following modules:

[0008] S1: Video stabilization module, which uses the scale-invariant feature transform (SIFT) algorithm to process video data to eliminate video instability caused by camera shaking, and uses the random sampling consensus (RANSAC) algorithm to remove mismatched points in feature point matching;

[0009] S2: vibration amplification module, which uses the Euler phase algorithm based on improved complex steerable pyramid (N-PBVM) to amplify the vibration of the video and realize vibration visualization;

[0010] S3: Vibration measurement module, which calculates the displacement data of the vibration measurement point in the video through the optical flow method of image phase correlation;

[0011] S4: a fault diagnosis module, which performs frequency domain analysis based on the data obtained by the vibration measurement module to identify the fault of the centrifugal pump;

[0012] S5: Data processing and storage module, used to store and further analyze the collected and processed vibration data.

[0013] The video stabilization module specifically includes:

[0014] The feature point extraction unit uses the SIFT algorithm to extract feature points from the video frame, specifically including:

[0015] A series of image points L(x, y, σ) are generated by Gaussian blur in different scale spaces. The formula is:

[0016] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0017] Among them, L(x, y, σ) is the image in different scale spaces, G(x, y, σ) is the Gaussian kernel function, I(x, y) is the input image, x and y are the spatial coordinates of the image, and σ is the scale factor.

[0018] Key points are detected by Gaussian difference (DoG), the formula is:

[0019]

[0020] Calculate the Gaussian difference image, the formula is:

[0021] D(x,y,σ)=L(x,y,kσ)-L(x,y,σ)

[0022] Where D(x, y, σ) is the Gaussian difference image, and k is the scale multiplication factor, which ranges from 1.2 to 1.6.

[0023] Calculate the gradient direction θ and magnitude m of the key point neighborhood, and assign the main direction and auxiliary direction to each key point. The formula is:

[0024]

[0025] Among them, m(x,y) is the gradient amplitude, θ(x,y) is the gradient direction, L(x+1,y) and L(x-1,y) represent the gradient of the image in the x direction, and L(x,y+1) and L(x,y-1) represent the gradient of the image in the y direction.

[0026] The feature point matching unit uses the RANSAC algorithm to remove mismatches in the feature point matching process;

[0027] The affine transformation unit calculates the affine transformation matrix between each frame of video and the initial frame, and realizes the mapping transformation of the feature point coordinates to the initial frame coordinate system.

[0028] The vibration amplification module specifically includes:

[0029] A video frame decomposition unit, which uses a complex steerable pyramid to perform multi-scale decomposition on the video frame;

[0030] The phase amplification unit performs phase amplification processing on the decomposed video frame. The formula is:

[0031] φ k (x,y,t)=φ0(x,y,t)+α·Δφ(x,y,t)

[0032] Among them, φ k (x, y, t) is the phase after amplification, φ0(x, y, t) is the initial phase, α is the amplification factor, and Δφ(x, y, t) is the phase change;

[0033] The video frame reconstruction unit reconstructs the amplified video frame to generate a vibration-amplified video.

[0034] The vibration amplification module further includes image reduction and image expansion steps, wherein the improved PBVM (N-PBVM) algorithm specifically includes:

[0035] An image reduction unit performs reduction processing on the input video image to reduce the computational complexity;

[0036] A spatial decomposition unit that uses a complex steerable pyramid to perform multi-scale decomposition on the reduced image;

[0037] A phase amplification unit, performing phase amplification processing on the decomposed video frame;

[0038] An image expansion unit expands the reduced image before pyramid reconstruction to restore the original image size and perform phase amplification;

[0039] The video frame reconstruction unit reconstructs the amplified video frame to generate a vibration-amplified video.

[0040] The vibration measurement module specifically includes:

[0041] An image phase calculation unit, which uses a two-dimensional Gabor filter to calculate the phase and amplitude of the video frame;

[0042]

[0043] Among them, x′=xcosθ+ysinθ, y′=-xsinθ+ycosθ, λ is the wavelength, θ is the directional angle, ψ is the phase shift, σ is the standard deviation of the Gaussian envelope, and γ is the spatial aspect ratio.

[0044] The displacement calculation unit calculates the pixel displacement in the video frame by the optical flow method related to the image phase. The formula is:

[0045]

[0046] Where v is the light loss, I is the image intensity;

[0047] The displacement conversion unit maps the pixel displacement to the actual physical displacement.

[0048] The fault diagnosis module specifically includes:

[0049] A data conversion unit, which converts vibration data from the time domain to the frequency domain using Fourier transform;

[0050] The fault identification unit identifies the fault characteristics of the centrifugal pump based on the frequency domain analysis results, including rotor imbalance, rotor misalignment and base looseness.

[0051] The data processing and storage module specifically includes:

[0052] A data storage unit, used for storing the collected and processed vibration data;

[0053] The data analysis unit uses machine learning algorithms to further analyze the stored data to predict failure trends.

[0054] A centrifugal pump fault diagnosis method based on visual sensing technology comprises the following steps:

[0055] Get video data of centrifugal pump operation;

[0056] The SIFT algorithm is used to extract feature points from video data, and the RANSAC algorithm is used to remove mismatched points;

[0057] The Euler phase algorithm based on complex steerable pyramid is used to amplify the vibration of the video.

[0058] Calculate the displacement data of the vibration measurement point in the video through the optical flow method of image phase correlation;

[0059] Perform frequency domain analysis on vibration data to identify fault characteristics of centrifugal pumps;

[0060] Vibration data is stored and further analyzed using machine learning algorithms to predict possible failure trends.

[0061] The feature point extraction region division technology divides the image into multiple regions and extracts feature points only in stable regions. Specifically, the image is divided into M×N grid regions. The formula is:

[0062]

[0063] Among them, G i,j represents the grid area at the i-th row and j-th column, W and H represent the width and height of the image respectively.

[0064] Extract feature points independently in each grid area to improve the accuracy of feature point extraction and avoid vibration interference of mechanical equipment. The formula is:

[0065] p=(x,y),d={d1,d2,...,d k}

[0066] Among them, p is the feature point position and d is the feature point descriptor.

[0067] The RANSAC algorithm is used to randomly select samples for fitting during the feature point matching process, specifically including selecting the initial matched feature point pairs {(p i ,q i )} randomly select n pairs of points, the formula is:

[0068] S={(p1,q1),(p2,q2),...,(p n ,q n )}

[0069] Calculate the model parameters and calculate the affine transformation matrix H by selecting the point pair S so that:

[0070]

[0071] Among them, the affine transformation matrix H is in the form of:

[0072]

[0073] Evaluate the error, for all matching point pairs {(p i ,q i )}Evaluate the error after transformation:

[0074] error(p i ,q i ,H)=||q i -Hp i ||

[0075] Among them, p i is the position of the feature point, q i are the corresponding matching points, is the point after affine transformation;

[0076] Filter out the inliers and take the point pairs whose error is less than the preset threshold ε as inliers:

[0077] {(p i ,q i |error(p i ,q i ,H)<ε}

[0078] Optimize the model through multiple iterations, repeat the above steps, iterate k times, and select the affine transformation matrix H with the most inliers. best As the final model, the mismatched points are removed and the matching accuracy is improved. The formula is:

[0079]

[0080] The affine transformation matrix calculation is used to maintain the geometric structure of the image, specifically including: according to the optimized feature point matching result, according to the optimized feature point matching result {(p i ,q i )}, calculate the affine transformation matrix H, so that:

[0081] q i =Hp i

[0082] Calculate the affine transformation matrix between each frame image and the initial frame, and transform the feature point coordinates (x i ,yi ), apply the affine transformation matrix H to transform the coordinates and obtain the transformed coordinates (x i ′,y i ′):

[0083]

[0084] Vibration spectrum analysis and time domain analysis are performed on the stabilized video, specifically including calculating the differential image between adjacent frames of the stabilized video. The formula is:

[0085] D t (x,y)=I t (x,y)-I t-1 (x,y)

[0086] Among them, D t (x, y) represents the difference image between the tth frame and the t-1th frame, I t (x,y) and I t-1 (x, y) represents the images of the tth frame and the t-1th frame respectively.

[0087] Extract vibration characteristics, identify the frequency components of vibration signals through spectrum analysis, and identify the time domain characteristics of vibration signals through time domain analysis, so as to perform fault diagnosis of mechanical equipment.

[0088] The beneficial effects of the present invention are as follows:

[0089] Non-contact measurement: The system uses visual sensing technology to achieve non-contact measurement of centrifugal pumps, avoiding the problems of complex installation, difficult maintenance and limited measurement range of traditional contact sensors.

[0090] Video stabilization: By using SIFT algorithm and RANSAC algorithm to process video data, the instability caused by camera shake can be effectively eliminated to ensure the accuracy and reliability of vibration measurement.

[0091] Vibration visualization: The vibration amplification module uses the improved Euler phase algorithm of complex steerable pyramid (N-PBVM), which can amplify the tiny vibrations in the video and realize vibration visualization, which is convenient for intuitive observation and analysis.

[0092] Efficient and accurate vibration measurement: The vibration measurement module calculates the vibration displacement in the video through the optical flow method of image phase correlation, ensuring the efficiency and accuracy of vibration measurement.

[0093] Fault diagnosis: The fault diagnosis module is based on frequency domain analysis and can accurately identify the fault characteristics of centrifugal pumps, including rotor imbalance, rotor misalignment and base looseness, thereby improving the accuracy of fault diagnosis.

[0094] Data processing and trend prediction: The data processing and storage module can not only store and analyze vibration data, but also use machine learning algorithms to predict failure trends, providing strong technical support for the maintenance of industrial equipment.

[0095] Through the above technical means, the present invention realizes efficient and reliable fault diagnosis of centrifugal pumps and provides advanced technical means for fault diagnosis and maintenance of industrial equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 :Flowchart of centrifugal pump fault diagnosis system based on visual sensing technology

[0097] This flowchart shows the workflow of the centrifugal pump fault diagnosis system based on visual sensing technology. The figure includes five main modules: video stabilization module, vibration amplification module, vibration measurement module, fault diagnosis module and data processing and storage module. Each module performs video data processing, vibration signal amplification and measurement, fault feature identification and data storage and analysis in turn, ultimately achieving accurate monitoring of the centrifugal pump operating status and fault prediction.

[0098] Figure 2 :SIFT feature point extraction diagram

[0099] This flowchart shows the specific steps of the SIFT algorithm to extract image feature points. First, the scale space is constructed and images of different scales are generated through Gaussian blur. Then, the Gaussian difference (DoG) is calculated to detect local extreme points and feature points. Next, the gradient direction and intensity of the feature points are calculated, and feature descriptors are generated. The whole process ensures that the extracted feature points have good scale invariance and rotation invariance, providing a basis for subsequent image matching and transformation.

[0100] Figure 3 : Original image before feature point extraction and matching

[0101] This image shows the original image of feature point extraction and matching before video stabilization. The left image is the first frame, and the right image is the 90th frame. The red circle represents the feature point extracted by the SIFT algorithm, and the yellow box is the feature point extraction area. In the original image, it can be seen that the distribution of feature points is dense and uneven, and some feature points may be affected by the vibration of the centrifugal pump.

[0102] Figure 4 : Stabilized image after feature point extraction and matching

[0103] This image shows the results of feature point extraction and matching after video stabilization. The left image is the first frame, and the right image is the 90th frame. The blue cross represents the feature point matched by the RANSAC algorithm, and the yellow box is the feature point extraction area. In the stabilized image, the feature points are more evenly distributed, the mismatched points have been removed, and the number of feature points has been significantly reduced, indicating that the accuracy and stability of matching have been significantly improved.

[0104] Figure 5 :Affine transformation of centrifugal pump and video stabilization effect diagram

[0105] This figure shows the effect of video processing of a centrifugal pump in operation. The left figure is an image after applying affine transformation, showing the result of feature point matching and affine transformation processing of video data by SIFT and RANSAC algorithms, which corrects the instability caused by camera shake. The right figure is an image after video stabilization processing, which makes the operation of the centrifugal pump more stable and smooth by eliminating jitter and other unstable factors. The feature points and marks in the picture are used to indicate the image alignment effect before and after processing, which helps to intuitively show the effect of affine transformation and video stabilization.

[0106] Figure 6 : Comparison of DITF values ​​before and after video stabilization

[0107] This figure shows the comparison of DITF (Dynamic Image Texture Feature) values ​​before and after video stabilization. The horizontal axis represents the number of video frames, and the vertical axis represents the DITF value (dB). The green curve represents the DITF value of the video before stabilization, and the blue curve represents the DITF value of the video after stabilization. It can be clearly seen that after stabilization, the DITF value of the video is significantly reduced, indicating that the dynamic texture features of the video are more stable. This result proves the effectiveness of the method of the present invention in reducing camera shake and improving video stability.

[0108] Figure 7 : Comparison of DITF average and standard deviation before and after video stabilization

[0109] This figure shows the comparison of the mean and standard deviation of the DITF (Dynamic Image Texture Feature) values ​​before and after the video stabilization process. The dark green bar graph represents the mean and standard deviation of the video DITF before stabilization, and the light green bar graph represents the mean and standard deviation of the video DITF after stabilization. It can be seen that after the stabilization process, the DITF mean value is significantly reduced and the standard deviation is also reduced, indicating that the dynamic texture features of the video are more stable and the fluctuations are reduced. This result further proves the significant effect of the method of the present invention in improving video stability. Example

[0110] This embodiment describes in detail the composition and working principle of a centrifugal pump operation video stabilization and vibration measurement system based on visual sensing technology.

[0111] Example 1: System composition and working principle

[0112] This embodiment describes in detail the composition and working principle of a centrifugal pump operation video stabilization and vibration measurement system based on visual sensing technology.

[0113] The system includes the following modules:

[0114] Video stabilization module

[0115] The scale-invariant feature transform (SIFT) algorithm is used to extract feature points from video frames. The random sample consensus (RANSAC) algorithm is used to remove mismatched points. The affine transformation matrix between each frame of video and the initial frame is calculated to achieve the mapping transformation of feature point coordinates to the initial frame coordinate system, thereby eliminating video instability caused by camera shaking.

[0116] Vibration Amplification Module

[0117] Including PBVM and N-PBVM algorithms:

[0118] PBVM algorithm: Multi-scale decomposition of video frames, phase amplification processing, and video frame reconstruction are performed through a complex steerable pyramid.

[0119] N-PBVM algorithm: Before decomposing the video frame, image reduction is performed first, followed by multi-scale decomposition and phase amplification, and finally image expansion and video frame reconstruction.

[0120] Vibration Measurement Module

[0121] The displacement data of the vibration measurement point in the video is calculated by the optical flow method related to the image phase. It includes an image phase calculation unit, a displacement calculation unit and a displacement conversion unit. The specific steps include:

[0122] A two-dimensional Gabor filter is used to calculate the phase and amplitude of the video frame;

[0123] Calculate the displacement vector of each pixel by optical flow method;

[0124] Convert pixel displacement vectors to actual physical displacements.

[0125] Fault diagnosis module

[0126] Frequency domain analysis is performed based on the data obtained from the vibration measurement module to identify the fault characteristics of the centrifugal pump, including rotor imbalance, rotor misalignment and base loosening.

[0127] Data processing and storage module

[0128] It is used to store and further analyze the collected and processed vibration data. It includes a data storage unit and a data analysis unit, and uses machine learning algorithms to further analyze the stored data to predict possible fault trends.

[0129] Embodiment 2: Video stabilization processing

[0130] Get the video data of the centrifugal pump running and input it into the video stabilization module. The specific steps of video stabilization processing are as follows:

[0131] Feature point extraction

[0132] The SIFT algorithm is used to extract feature points from each frame of video image. The SIFT algorithm constructs a scale space to detect and describe local feature points in the image, ensuring that the feature points have good scale invariance and rotation invariance.

[0133] The SIFT algorithm is used to extract feature points from video frames, including:

[0134] A series of images L(x, y, σ) are generated by Gaussian blur in different scale spaces. The formula is:

[0135] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0136] Among them, G(x,y,σ) is the Gaussian kernel function and I(x,y) is the input image.

[0137] Feature point matching

[0138] Key points are detected by Gaussian difference (DoG), the formula is:

[0139]

[0140] Calculate the Gaussian difference image, the formula is:

[0141] D(x,y,σ)=L(x,y,kσ)-L(x,y,σ)

[0142] Among them, k is the scale multiplication factor.

[0143] Affine transformation calculation

[0144] Calculate the gradient direction θ and magnitude m of the key point neighborhood, and assign the main direction and auxiliary direction to each key point. The formula is:

[0145]

[0146] The image is divided into multiple regions, and feature points are extracted only in stable regions. Specifically, the image is divided into M×N grid regions, and the formula is:

[0147]

[0148] Among them, G i,j represents the grid area at the i-th row and j-th column, W and H represent the width and height of the image respectively.

[0149] Extract feature points independently in each grid area to improve the accuracy of feature point extraction and avoid vibration interference of mechanical equipment. The formula is:

[0150] p=(x,y),d={d1,d2,...,d k}

[0151] Among them, p is the feature point position and d is the feature point descriptor.

[0152] The RANSAC algorithm is used to randomly select samples for fitting during the feature point matching process, specifically including selecting the initial matched feature point pairs {(p i ,q i )} randomly select n pairs of points, the formula is:

[0153] S={(p1,q1),(p2,q2),...,(p n ,q n )}

[0154] Calculate the model parameters and calculate the affine transformation matrix H by selecting the point pair S so that:

[0155]

[0156] Among them, the affine transformation matrix H is in the form of:

[0157]

[0158] Evaluate the error, for all matching point pairs {(p i ,q i )}Evaluate the error after transformation:

[0159] error(p i ,q i ,H)=||q i -Hp i ||

[0160] Among them, p i is the position of the feature point, q i are the corresponding matching points, is the point after affine transformation;

[0161] Filter out the inliers and take the point pairs whose error is less than the preset threshold ε as inliers:

[0162] {(p i ,q i |error(p i ,q i ,H)<ε}

[0163] Optimize the model through multiple iterations, repeat the above steps, iterate k times, and select the affine transformation matrix H with the most inliers. best As the final model, the mismatched points are removed and the matching accuracy is improved. The formula is:

[0164]

[0165] The affine transformation matrix calculation is used to maintain the geometric structure of the image, specifically including the optimized feature point matching results, according to the optimized feature point matching results {(p i ,q i )}, calculate the affine transformation matrix H, so that:

[0166] q i =Hp i

[0167] Calculate the affine transformation matrix between each frame image and the initial frame, and transform the feature point coordinates (x i ,y i ), apply the affine transformation matrix H to transform the coordinates and obtain the transformed coordinates (x i ′,y i ′):

[0168]

[0169] The stabilized video is subjected to vibration spectrum analysis and time domain analysis, specifically including calculating the differential image between adjacent frames of the stabilized video. The formula is:

[0170] D t (x,y)=I t (x,y)-I t-1 (x,y)

[0171] Among them, D t (x, y) represents the difference image between the tth frame and the t-1th frame, I t (x,y) and I t-1 (x, y) represents the images of the tth frame and the t-1th frame respectively.

[0172] Example 3: Vibration Amplification and Measurement

[0173] The stabilized video is subjected to vibration amplification and measurement processing. The specific steps are as follows:

[0174] Vibration Amplification

[0175] Select PBVM or N-PBVM algorithm for vibration amplification processing:

[0176] PBVM algorithm: uses a complex steerable pyramid to perform multi-scale decomposition and phase amplification processing on video frames.

[0177] The phase amplification processing formula is:

[0178] φ k (x,y,t)=φ0(x,y,t)+α·Δφ(x,y,t)

[0179] Among them, φ k (x, y, t) is the phase after amplification, φ0(x, y, t) is the initial phase, α is the amplification factor, and Δφ(x, y, t) is the phase change;

[0180] N-PBVM algorithm: first perform image reduction, then perform multi-scale decomposition and phase amplification, and finally perform image expansion and video frame reconstruction.

[0181] Vibration Measurement

[0182] The displacement data of the vibration measurement point in the video is calculated by the optical flow method of image phase correlation. The specific steps include:

[0183] The two-dimensional Gabor filter is used to calculate the phase and amplitude of the video frame. The formula is:

[0184]

[0185] Among them, x′=xcosθ+ysinθ, y′=-xsinθ+ycosθ.

[0186] The pixel displacement in the video frame is calculated by the optical flow method related to the image phase, and the formula is:

[0187]

[0188] Where v is the light loss, I is the image intensity;

[0189] The formula for converting the pixel displacement vector into actual physical displacement is:

[0190] Δx physical =Δx pixel ·S

[0191] Where Δx physical is the actual physical displacement, Δx pixelis the pixel displacement and S is the scale factor.

[0192] Example 4: Fault diagnosis and data analysis

[0193] The data obtained by the vibration measurement module is used for fault diagnosis and further analysis. The specific steps are as follows:

[0194] Frequency Domain Analysis

[0195] The vibration data is analyzed in the frequency domain and converted from the time domain to the frequency domain using Fourier transform to identify the fault characteristics of the centrifugal pump, including rotor imbalance, rotor misalignment and base looseness.

[0196] Perform Fourier transform on the time domain signal x(t) and convert it into the frequency domain signal X(f). The formula is:

[0197]

[0198] Among them, f(t) represents the time domain signal and F(ω) represents the frequency domain signal.

[0199] Data Storage and Analysis

[0200] The vibration data is stored in the data processing and storage module, and the stored data is further analyzed using machine learning algorithms to predict possible failure trends and provide decision support for equipment maintenance and management.

[0201] The stored data is preprocessed, including denoising, normalization, and feature extraction, to generate a data set suitable for machine learning model input.

[0202] Select appropriate machine learning algorithms (such as support vector machines, decision trees, neural networks, etc.), train the preprocessed data, and generate fault prediction models.

[0203] The trained machine learning model is used to analyze the real-time collected vibration data and predict possible failure trends. The formula is:

[0204]

[0205] in, is the predicted fault state, z is the input feature vector, and f is the trained machine learning model.

[0206] In order to verify the effectiveness of the video stabilization and vibration measurement method based on visual sensing technology in the present invention in the fault diagnosis of centrifugal pumps, the following experiments were conducted and significant results were achieved. The experimental environment includes an AMD Ryzen 75800H processor, 16GB memory and an NVIDIA GeForce RTX3060 graphics card, the operating system is Windows 10, and the development tools are Python 3.8 and OpenCV library. The video acquisition equipment includes a high-resolution and high-frame-rate camera, a highly adaptable light source and a high-speed image acquisition card. The test objects include handheld static scene videos and videos recording the operation of the centrifugal pump.

[0207] The experimental steps are as follows: First, videos were recorded from the handheld device and the centrifugal pump during operation with a resolution of 3840×2160 and a frame rate of 30fps, and segmented into single-frame images, with 180 frames extracted from each video. Next, feature points were extracted from the video frames using the SIFT algorithm, Gaussian blurred images were generated in different scale spaces, and key points were detected using the Difference of Gaussians (DoG). Feature point extraction region partitioning technology was used to extract feature points only in stable regions. The RANSAC algorithm was used to remove mismatched points, and the affine transformation matrix between each frame of video and the initial frame was calculated to achieve the mapping transformation of feature point coordinates to the initial frame coordinate system. Then, the video was stabilized to correct the image displacement caused by camera shake.

[0208] In the vibration amplification and measurement part, the PBVM or N-PBVM algorithm is selected for vibration amplification processing, and the complex steerable pyramid is used to perform multi-scale decomposition and phase amplification processing on the video frame. The displacement data of the vibration measurement points in the video is calculated by the optical flow method of image phase correlation, and the phase and amplitude of the video frame are calculated by a two-dimensional Gabor filter. The stabilized video is subjected to vibration spectrum analysis and time domain analysis to extract vibration features. The spectrum analysis uses Fourier transform to convert the vibration data from the time domain to the frequency domain to identify the fault characteristics of the centrifugal pump, and the time domain analysis extracts the peak, average and standard deviation of the characteristic signal.

[0209] The experimental results show that the average number of feature points extracted from each frame of the original video frame is 500. After processing, the average number of valid feature points per frame is 200, the initial number of matching pairs is 450, and the number of valid matching pairs after RANSAC removes false matches is 180. The RANSAC algorithm significantly improves the accuracy of feature matching from 92.15% to 100%. The accuracy of affine transformation matrix calculation reaches more than 99%, which can effectively correct the image displacement caused by camera shake. The calculation time of affine transformation matrix is ​​0.05 seconds per frame on average, which meets the real-time processing requirements. In the video after stabilization, the image displacement caused by camera shake is completely corrected, the image stability is significantly improved, the DITF (inter-frame transform fidelity) average value and standard deviation of the stabilized video are significantly reduced, and the video stability is significantly improved. In terms of vibration measurement and fault diagnosis, the spectrum analysis results show that the main vibration frequency components are accurately identified, and the time domain analysis results show that the time domain features such as the peak value, average value and standard deviation of the characteristic signal are accurately extracted.

[0210] Through the above experimental process, the video stabilization method based on SIFT feature point extraction and RANSAC algorithm provided by the present invention can effectively solve the video instability problem caused by camera shaking, and provide a stable and reliable data source for vibration measurement and fault diagnosis of industrial equipment. The experimental results show that this method has high stability and accuracy in practical applications.

Claims

1. A centrifugal pump fault diagnosis system based on visual sensing technology, characterized in that: Includes the following modules: S1: Video stabilization module, which uses a scale-invariant feature transformation algorithm to process video data to eliminate video instability caused by camera shake and removes false matching points in feature point matching through a random sampling consistency algorithm; S2: vibration amplification module, which uses the Euler phase algorithm based on the improved complex steerable pyramid to amplify the video vibration and realize vibration visualization; S3: Vibration measurement module, which calculates the displacement data of vibration measurement points in the video through the optical flow method of image phase correlation; S4: a fault diagnosis module, which performs frequency domain analysis based on the data obtained by the vibration measurement module to identify the fault of the centrifugal pump; S5: Data processing and storage module, used to store and further analyze the collected and processed vibration data.

2. The centrifugal pump fault diagnosis system based on visual sensing technology according to claim 1, wherein the video stabilization module specifically comprises: The feature point extraction unit uses a scale-invariant feature transformation algorithm to extract feature points from video frames. Specifically, it includes: A series of image points L(x, y, σ) are generated by Gaussian blur in different scale spaces. The formula is: L(x,y,σ)=G(x,y,σ)*I(x,y) Where L(x,y,σ) is the image in different scale spaces, G(x,y,σ) is the Gaussian kernel function, I(x,y) is the input image, x and y are the spatial coordinates of the image, and σ is the scale factor; The key points are detected by Gaussian difference, and the formula is: Calculate the Gaussian difference image, the formula is: D(x,y,σ)=L(x,y,kσ)-L(x,y,σ) Where D(x,y,σ) is the Gaussian difference image, k is the scale multiplication factor, and its value range is between 1.2 and 1.6; Calculate the gradient direction θ and amplitude m of the key point neighborhood, and assign the main direction and auxiliary direction to each key point. The formula is: Where m(x,y) is the gradient magnitude, θ(x,y) is the gradient direction, L(x+1,y) and L(x-1,y) represent the gradient of the image in the x direction, and L(x,y+1) and L(x,y-1) represent the gradient of the image in the y direction. Feature point matching unit, which uses random sampling consistency algorithm to remove false matches in the feature point matching process; The affine transformation unit calculates the affine transformation matrix between each frame of video and the initial frame, and realizes the mapping transformation of the feature point coordinates to the initial frame coordinate system.

3. According to the centrifugal pump fault diagnosis system based on visual sensing technology in claim 1, the vibration amplification module specifically comprises: Video frame decomposition unit, which uses a complex steerable pyramid to perform multi-scale decomposition of video frames; The phase amplification unit performs phase amplification processing on the decomposed video frame. The formula is: φ k (x,y,t)=φ0(x,y,t)+α·Δφ(x,y,t) Among them, φ k (x, y, t) is the phase after amplification, φ0(x, y, t) is the initial phase, α is the amplification factor, and Δφ(x, y, t) is the phase change; The video frame reconstruction unit reconstructs the amplified video frame to generate a vibration-amplified video.

4. The centrifugal pump fault diagnosis system based on visual sensing technology according to claim 1, wherein the vibration amplification module further comprises image reduction and image expansion steps, characterized in that: The improved vibration measurement algorithm specifically includes: An image reduction unit performs reduction processing on the input video image to reduce computational complexity; A spatial decomposition unit that uses a complex steerable pyramid to perform multi-scale decomposition on the reduced image; A phase amplification unit, performing phase amplification processing on the decomposed video frames; An image expansion unit that expands the reduced image before pyramid reconstruction to restore the original image size and perform phase amplification; The video frame reconstruction unit reconstructs the amplified video frame to generate a vibration-amplified video.

5. The centrifugal pump fault diagnosis system based on visual sensing technology according to claim 1, wherein the vibration measurement module specifically comprises: Image phase calculation unit, which uses a two-dimensional Gabor filter to calculate the phase and amplitude of the video frame; Where x′ = xcosθ + ysinθ, y′ = -xsinθ + ycosθ, λ is the wavelength, θ is the azimuth angle, ψ is the phase offset, σ is the standard deviation of the Gaussian envelope, and γ is the spatial aspect ratio; The displacement calculation unit calculates the pixel displacement in the video frame through the optical flow method of image phase correlation. The formula is: Where v is the light loss, I is the image intensity; The displacement conversion unit maps pixel displacement to actual physical displacement.

6. The centrifugal pump fault diagnosis system based on visual sensing technology according to claim 1, wherein the fault diagnosis module specifically comprises: A data conversion unit converts vibration data from the time domain to the frequency domain using Fourier transform; The fault identification unit identifies the fault characteristics of the centrifugal pump based on the frequency domain analysis results, including rotor imbalance, rotor misalignment and base looseness.

7. The centrifugal pump fault diagnosis system based on visual sensing technology according to claim 1, wherein the data processing and storage module specifically comprises: A data storage unit, used for storing collected and processed vibration data; The data analysis unit uses machine learning algorithms to further analyze the stored data to predict failure trends.

8. A centrifugal pump fault diagnosis method based on visual sensing technology, characterized in that: The following steps are involved: Obtain video data of centrifugal pump operation; The scale-invariant feature transformation algorithm is used to extract feature points from video data, and the random sampling consistency algorithm is used to remove mismatched points. The Euler phase algorithm based on complex steerable pyramid is used to amplify the vibration of the video. Calculate the displacement data of vibration measurement points in the video through the optical flow method of image phase correlation; Perform frequency domain analysis on vibration data to identify fault characteristics of centrifugal pumps; Vibration data is stored and further analyzed using machine learning algorithms to predict possible failure trends.

9. The method according to claim 7, characterized in that The feature point extraction region division technology divides the image into multiple regions and extracts feature points only in stable regions. Specifically, the image is divided into M×N grid regions. The formula is: Among them, G i,j represents the grid area at row i and column j, W and H represent the width and height of the image respectively; Feature points are extracted independently in each grid area to improve the accuracy of feature point extraction and avoid interference from mechanical equipment vibration. The formula is: p=(x,y),d={d1,d2,...,d k } Among them, p is the feature point position and d is the feature point descriptor.

10. The centrifugal pump fault diagnosis method based on visual sensing technology according to claim 7, characterized in that: The RANSAC algorithm is used to randomly select samples for fitting during the feature point matching process, specifically including in each iteration, selecting the initial matched feature point pairs {(p i ,q i )} randomly select n pairs of points, the formula is: S={(p1,q1),(p2,q2),...,(p n ,q n )} Calculate the model parameters and calculate the affine transformation matrix H by selecting the point pair S so that: Among them, the affine transformation matrix H is in the form of: Evaluate the error, for all matching point pairs {(p i ,q i )}Evaluate the error after transformation: error(p i ,q i ,H)=||q i -Hp i || Among them, p i is the position of the feature point, q i is the corresponding matching point, H pi is the point after affine transformation; Filter out the inliers and take the point pairs whose error is less than the preset threshold ε as inliers: {(p i ,q i |error(p i ,q i ,H)<ε} Optimize the model through multiple iterations, repeat the above steps, iterate k times, and select the affine transformation matrix H with the most inliers. best As the final model, the mismatched points are removed and the matching accuracy is improved. The formula is:

11. The centrifugal pump fault diagnosis method based on visual sensing technology according to claim 10, characterized in that: The affine transformation matrix calculation is used to maintain the geometric structure of the image, specifically including according to the optimized feature point matching result, according to the optimized feature point matching result {(p i ,q i )}, calculate the affine transformation matrix H so that: q i =Hp i Calculate the affine transformation matrix between each frame image and the initial frame, and calculate the feature point coordinates (x i ,y i ), apply the affine transformation matrix H to perform coordinate transformation and obtain the transformed coordinates (x i ′,y i ′):

12. The centrifugal pump fault diagnosis method based on visual sensing technology according to claim 11, characterized in that: Vibration spectrum analysis and time domain analysis are performed on the stabilized video. Specifically, the differential image between adjacent frames of the stabilized video is calculated using the formula: D t (x,y)=I t (x,y)-I t-1 (x,y) Among them, D t (x, y) represents the difference image between the t-th frame and the t-1-th frame, I t (x,y) and I t-1 (x, y) represent the images of the t-th frame and the t-1-th frame respectively; Extract vibration characteristics, identify the frequency components of vibration signals through spectrum analysis, and identify the time domain characteristics of vibration signals through time domain analysis, so as to perform fault diagnosis of mechanical equipment.

Images