Visual real-time vibration monitoring algorithm and system

Abstract

The invention belongs to the technical field of vibration monitoring, and particularly relates to a visual real-time vibration monitoring algorithm and system.The method comprises the steps that image acquisition devices are deployed at different positions of equipment to be detected, and image information of the positions to be detected of the equipment to be detected is obtained through the image acquisition devices; the method comprises the following steps: acquiring image information of a to-be-detected position of to-be-detected equipment to obtain vibration information of the to-be-detected position, and converting an analog signal of the vibration information into digital information; performing data analysis on the digital information of the position to be detected based on a visual real-time vibration monitoring algorithm; the vibration frequency and the vibration amplitude of the to-be-tested position of the to-be-tested equipment are obtained based on data analysis; and generating real-time visual reports of different positions of the to-be-tested equipment according to the obtained vibration frequency and vibration amplitude of the to-be-tested position of the to-be-tested equipment. The monitoring process is not affected by environment temperature, humidity and electromagnetic interference, real-time monitoring of vibration is achieved, a data processing center does not need a large amount of storage space and computing resources, and labor and cost are greatly reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of vibration monitoring, and in particular relates to a visualized real-time vibration monitoring algorithm and system. Background Art

[0002] Key industrial equipment and objects will produce varying degrees of vibration during operation. These vibrations may be caused by the structural characteristics of the equipment itself, operating parameters, external interference and other factors. If the vibration exceeds the normal range, it may cause equipment damage, failure or accident, which may have serious consequences in industry. Therefore, it is very necessary and important to monitor and analyze the vibration of key equipment and machines in real time, so as to timely discover and prevent potential problems and improve the reliability and life of the equipment. The main methods for vibration monitoring in the prior art are as follows: One method is to use an accelerometer or other type of sensor fixed on the target object, collect its vibration signal, and transmit it to the data processing center for analysis via wired or wireless means. The advantage of this method is that it can realize continuous collection and real-time analysis of vibration signals, but on the one hand, the installation of the sensor requires equipment space, which may affect the normal operation of the equipment; on the other hand, the sensor itself will also be affected by factors such as ambient temperature, humidity, and electromagnetic interference, resulting in a decrease in signal quality; on the other hand, due to the limited number of sensors, it cannot cover all parts where abnormal vibration may occur; finally, the data transmission method may have problems such as delay, loss, and interference, and the data processing center requires a large amount of storage space and computing resources.

[0003] Another method is to use handheld or portable vibration detection instruments to perform regular or irregular detection of target objects. The advantage of this method is that the detection time and location can be flexibly selected, and different parts can be focused on as needed. However, the detection process requires manual operation, which consumes manpower and material resources; the detection frequency and coverage are limited by factors such as personnel arrangements and the number of equipment; and the detection results need to be manually judged and recorded, which is prone to subjective errors and omissions; the detection results cannot be fed back to relevant personnel and departments in a timely manner, making it difficult to achieve rapid response and processing.

[0004] Therefore, the space occupied by installing sensors in the prior art may affect the normal operation of the equipment. The sensors themselves may also be affected by factors such as ambient temperature, humidity, and electromagnetic interference, resulting in a decrease in signal quality. In addition, the sensors cannot cover all parts where abnormal vibrations may occur, and the data transmission method may have problems such as delays, losses, and interference. The data processing center requires a large amount of storage space and computing resources. The handheld or portable vibration detection instruments have low detection efficiency, waste of manpower and material resources, and large errors, which are technical problems that need to be solved urgently. Summary of the invention

[0005] The purpose of the present invention is to provide a visual real-time vibration monitoring algorithm and system to solve the problems that the installation of sensors occupies equipment space, which may affect the normal operation of the equipment. The sensors themselves may be affected by factors such as ambient temperature, humidity, electromagnetic interference, etc., resulting in a decrease in signal quality and failure to cover all parts where abnormal vibrations may occur. In addition, the data transmission method may have delays, losses, interference, etc. The data processing center requires a large amount of storage space and computing resources. Handheld or portable vibration detection instruments have low detection efficiency, waste of manpower and material resources, and large errors.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows: In the first aspect, a visual real-time vibration monitoring algorithm is provided, comprising the following steps: S1: deploying image acquisition devices at different positions of the device to be tested, and acquiring image information of the position to be tested of the device to be tested through the image acquisition devices; S2: obtaining vibration information of the position to be tested by acquiring image information of the position to be tested of the device to be tested, and converting an analog signal of the vibration information into a digital signal; S3: Data analysis of digital signals at the measured position based on real-time vibration monitoring algorithm based on visualization; S4: Obtaining the vibration frequency and vibration amplitude of the position to be tested of the device to be tested based on data analysis and calculation; S5: Generate a real-time visual report of different positions of the device under test using the obtained vibration frequency and vibration amplitude of the device under test at the position under test.

[0007] Preferably, step S1 includes the following specific process: S11: Mark the designated position of the device to be tested, that is, mark the key point; S12: The image information of key points of the device to be tested is collected by an image acquisition device as a basis for vibration data of the device to be tested.

[0008] Preferably, step S3 includes the following specific process: S31: Detect key points of the device to be tested using image information of the key points; S32: Use keypoint matching to track the new position frame by frame; S33: Build a vibration monitoring model and perform data analysis on key points.

[0009] Preferably, step S31 includes the following specific process: S311: graying the image by weighted average method. According to the different sensitivity of human eyes to different colors, the green channel has the highest weight, the red channel has the second highest weight, and the blue channel has the lowest weight. Threshold segmentation is performed by binarization method. S312: Detect and extract feature vectors of key points from the binary image; S313: Establishing a PN template dictionary; S314: Determine the category of the corresponding key point by calculating and comparing the similarity score between the PN template dictionary and the feature vector.

[0010] Preferably, the formula of the weighted average method in step S311 is: ; in, Represents the grayscale image after grayscale conversion, R, G, and B represent the red, green, and blue channels of the color image respectively; The binarization method uses a fixed global threshold T to compare each pixel in the grayscale image with the global threshold T. When the grayscale value is less than the preset threshold T, the image is set to white and all three channels are set to 255; otherwise, it is set to black and all three channels are set to 0. The binarization formula is as follows: ; Where B(x, y) is the pixel value at coordinates (x, y) in the binary image, Gray(x, y) is the pixel value at coordinates (x, y) in the grayscale image, and T represents a custom threshold.

[0011] Preferably, the feature vector of the key point in step S312 is a curve whose outline is composed of points connected by the same gray value, representing the shape and boundary of the object, or the descriptor of the key point of the current frame is detected according to different retrieval modes and approximation methods, and all the key points detected in the current frame generate an 11-dimensional feature vector ; The optimal feature vector V It is composed of the following elements: V ={x coordinate of the key point center, y coordinate of the key point center, minimum enclosing rectangle border length, minimum enclosing rectangle border width, number of all points on the key point contour, contour hierarchy information, average pixel inside the key point contour, average pixel around the feature point contour extending 10 pixels outward}; The hierarchical information of the contour is a vector of length 4, which respectively represents the index of the next, previous, first child node and parent node of the current contour.

[0012] Preferably, the similarity score expression in step S314 is as follows: ; in, V i It is i The feature vector of the key point, D P，N Indicates that the template dictionary is V i The feature vector of the most recent sample, α , β , δ is the weight coefficient, satisfying ; KL(·) is the KL divergence, which is used to measure the difference between two probability distributions. The expression for measuring the difference between two probability distributions is as follows: ; Among them, P and Q are two discrete probability distributions, x is a possible value, log is the natural logarithm; the expression that measures the difference between two probability distributions means calculating the amount of information required to approximate distribution P from distribution Q, that is, the loss of information entropy; the larger the KL divergence, the less similar the two distributions are; ED(•) is the Euclidean distance, which is used to measure the spatial distance between two vectors. The Euclidean distance expression is as follows: ; in, u and v are two n-dimensional vectors, u i and v i They are in the i The Euclidean distance expression means to calculate the straight-line distance between two vectors in Euclidean space, which is the generalization of the Pythagorean theorem. The smaller the Euclidean distance, the closer the two vectors are. CS(·) is the cosine similarity, which is used to measure the directional similarity of two vectors; the cosine similarity expression is as follows: ; in, u and v There are two n dimensional vector, u i and v i They are in the i The cosine similarity expression means to calculate the cosine value of the angle between two vectors, that is, the inner product between their projections on the unit circle; the closer the cosine similarity is to 1, the more parallel the two vectors are; the closer the cosine similarity is to -1, the more opposite the two vectors are; the cosine similarity is 0, which means the two vectors are orthogonal.

[0013] Preferably, the specific process of using key point matching to track the new position frame by frame in step S32 is as follows: S321: Create a corresponding tracker list based on the detected key points To store the displacement time series of the frames before and after the key points, Indicates t Frame n The tracker of key points stores the nth key point t The coordinate sequence of the frame, Indicates n The key point is t Frame x , y Axis coordinates; S322: For each tracker of each frame, a preset algorithm is used to match the optimal displacement between the key points of the previous and next frames; The key points of the previous and next frames are matched by the tracker coordinates of the previous frame and the coordinates of the key points of the current frame and the similarity measurement of the corresponding type. The expression is as follows: ; in, T i The previous frame i The coordinates of the trackers, K j The current frame j The coordinates of the key points, y i and y j are the corresponding key point types, s is the key point scale, ED is the Euclidean distance, which is used to measure the spatial distance between two coordinates. kd is the Kronecker function, which is used to determine whether two types are the same. If they are the same, it is 1, otherwise it is 0. The expression comprehensively measures the spatial proximity and semantic consistency of the key points in the previous and next frames to obtain a similarity value. The higher the similarity value, the more matched it is, and the lower the similarity value, the less matched it is. S323: When n Trackers are matched successfully, and the coordinate sequence of the key points that are currently matched successfully is appended Tracking in the tracker, the formula is: ; S324: Repeat S322 and S323 until all trackers of the previous frame are matched or all key points of the current frame are matched; S325: For a tracker that does not track displacement or loses key points due to occlusion or blur in the image, a Kalman filter is used to predict the most likely position of the lost key point, and the most similar candidate key point is searched near the position. When the most similar candidate key point is found, the tracker is updated, otherwise the prediction state is maintained until a match is found or the maximum allowed number of lost frames is exceeded; Alternatively, the optical flow method is used to estimate the direction and speed of the lost key point, and the most similar candidate key point is found along the motion trajectory in that direction. When the most similar candidate key point is found, the tracker is updated, otherwise the motion state is maintained until a match is found or the maximum allowed number of lost frames is exceeded; Alternatively, a neural network model that can identify and recover lost key points is trained through deep learning methods, and the model is used to generate the most likely candidate key points in the current frame. If the generation is successful, the tracker is updated, otherwise the generation state is maintained until the generation matches or the maximum allowed number of lost frames is exceeded.

[0014] Preferably, the real-time vibration monitoring algorithm of step S3 includes a Fourier transform algorithm, a wavelet transform algorithm, and a power spectrum density estimation algorithm; The calculation process of the vibration amplitude in step S4 is as follows: S41: Select a feature point and measure the length l of the feature point in the image in pixels; S42: Calculate the displacement d of the feature point in the horizontal direction and the vertical direction by comparing images of different time series; S43: Calculate the vibration amplitude by the displacement d of the feature point in the horizontal and vertical directions: Using the principle of similar triangles, the actual size s of the image in millimeters; the resolution r of the image in pixels / mm, calculate the actual displacement of the feature point in millimeters, that is, the vibration amplitude, by the formula: A = d*s / r; Finally, the vibration amplitude A is obtained, which is the maximum deviation distance of the target object in the actual space; Alternatively, the horizontal distance d between the camera and the target and the angle θ between the line connecting the camera and the target and the horizontal direction can be obtained through a laser or hardware distance measuring device; the displacement A of the target can be calculated using the relationship between trigonometric functions, that is: A=d*tan(θ); Finally, the vibration amplitude A is obtained, which is the maximum deviation distance of the target object in the actual space; Or obtain the actual vibration amplitude of the object through a displacement sensor; Or use a three-axis acceleration sensor or a three-dimensional laser scanner to measure a certain position of the target object and obtain the position x axis,y Axis and z The vibration information on the axis is decomposed and synthesized through the data processing module to obtain the distribution and changes of the vibration amplitude and frequency in different directions at that position.

[0015] In a second aspect, a visual real-time vibration monitoring system is provided, comprising: an image acquisition device, a data processing module and a visualization module; The image acquisition device is used to acquire image information of a specified position of the device to be tested; The data processing module is used to analyze the data containing vibration information and calculate the vibration frequency and amplitude; The visualization module is used to generate and display real-time visualization reports based on the output of the data processing module; wherein the visualization reports are in the form of charts, curves, images, videos and sounds.

[0016] The beneficial effects of the present invention include: The visualized real-time vibration monitoring algorithm and system provided by the present invention, on the one hand, avoid installing sensors on the equipment to be tested, reduce space occupation, and will not affect the normal operation of the equipment. At the same time, it avoids the vibration monitoring results being affected by factors such as ambient temperature, humidity, and electromagnetic interference, thereby improving the accuracy of the monitoring results; on the other hand, it can cover all parts where abnormal vibrations may occur, and realize real-time monitoring of the vibrations of each monitoring point of the equipment to be tested, without the problems of data delay, loss, interference, etc.; on the other hand, the data processing center does not need a large amount of storage space and computing resources, and at the same time solves the technical problems of low detection efficiency, waste of manpower and material resources, and large errors existing in handheld or portable vibration detection instruments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the visualized real-time vibration monitoring algorithm of the present invention.

[0018] Figure 2 This is a schematic diagram of a visual report of the present invention. DETAILED DESCRIPTION

[0019] The following is combined with Figure 1-Figure 2 The present invention is further described in detail: Example 1 See also Figure 1 , a visual real-time vibration monitoring algorithm, including the following steps: S1: deploying image acquisition devices at different positions of the device to be tested, and acquiring image information of the position to be tested of the device to be tested through the image acquisition devices; S2: obtaining vibration information of the position to be tested by acquiring image information of the position to be tested of the device to be tested, and converting an analog signal of the vibration information into a digital signal; S3: Data analysis of digital signals at the measured position based on real-time vibration monitoring algorithm based on visualization; S4: Obtaining the vibration frequency and vibration amplitude of the position to be tested of the device to be tested based on data analysis and calculation; S5: Generate a real-time visual report of different positions of the device under test using the obtained vibration frequency and vibration amplitude of the device under test at the position under test.

[0020] By deploying image acquisition devices at different positions of the device to be tested, image information of the position to be tested of the device to be tested is collected, and based on the image information, vibration information of the position to be tested is obtained, and the vibration information is converted into a digital signal. The target object can be photographed using a lens to obtain a video or picture sequence of the target object and convert it into a digital signal. The target object can also be scanned using lasers, sound waves, infrared rays, etc. to obtain the reflection signal of the target object and convert it into a digital signal. The target object can also be measured using an accelerometer, a displacement sensor, etc. to obtain the motion signal of the target object and convert it into a digital signal. In addition, multiple cameras can be used to photograph or measure different positions of the target object to obtain vibration information at different positions. For the vibration of the same position on different axes, a three-axis accelerometer or a three-dimensional laser scanner can be used to measure the target object to obtain the vibration information of the position on the x-axis, y-axis and z-axis, and a comprehensive analysis can be performed through a data processing module to obtain the overall vibration characteristics and local vibration anomalies of the target object.

[0021] The data containing vibration information is analyzed by one or more algorithms, and the vibration frequency and amplitude are calculated. The data containing vibration information is analyzed in the frequency domain by Fourier transform to obtain the vibration spectrum, and the main vibration frequency and amplitude are extracted from it. Alternatively, the data containing vibration information is analyzed in multiple scales by wavelet transform to obtain the vibration characteristics at different scales, and the main vibration frequency and amplitude are extracted from it. Alternatively, the data containing vibration information is analyzed by power spectrum density estimation to obtain the vibration power spectrum, and the main vibration frequency and amplitude are extracted from it. The data processing module is used to decompose and synthesize, and the distribution and change of the vibration amplitude and frequency of the target object in different directions are obtained.

[0022] Based on the calculated vibration frequency and vibration amplitude, a real-time visual report is generated and displayed to the user. The user can view the vibration status of the target object in real time through a graphical interface. The distribution of vibration amplitude and frequency is displayed by using waveforms, bar graphs, pie charts, etc., and abnormal or dangerous areas are marked by color, size, shape, etc. Or use curve graphs, scatter plots, heat maps, etc. to display the changes in vibration amplitude and frequency over time or space, and use sound, text, icons, etc. to remind users to pay attention or take measures. Or use three-dimensional models, virtual reality, augmented reality, etc. to display the real-time status of the target object, and use animation, special effects, interaction, etc. to enhance the user experience. Or display the vibration conditions of different positions or different axes in the form of graphs, tables, etc., and use sorting, filtering, comparison and other functions to help users analyze vibration data.

[0023] In this embodiment, step S1 includes the following specific processes: S11: Mark the designated position of the device to be tested, that is, mark the key point; S12: The image information of key points of the device to be tested is collected by an image acquisition device as a basis for vibration data of the device to be tested.

[0024] Example 2 Based on Example 1, step S3 includes the following specific process: S31: Detect key points of the device to be tested using image information of the key points; S32: Use keypoint matching to track the new position frame by frame; S33: Build a vibration monitoring model and perform data analysis on key points.

[0025] Step S31 includes the following specific processes: S311: grayscale the image using a weighted average method. According to the different sensitivities of the human eye to different colors, the green channel has the highest weight, the red channel has the second highest weight, and the blue channel has the lowest weight. Threshold segmentation is performed using a binarization method.

[0026] S312: Detect and extract feature vectors of key points from the binary image; S313: Establishing a PN template dictionary; S314: Determine the category of the corresponding key point by calculating and comparing the similarity score between the PN template dictionary and the feature vector.

[0027] The formula of the weighted average method in step S311 is: ; in, Represents the grayscale image after grayscale conversion, R, G, and B represent the red, green, and blue channels of the color image respectively; The binarization method uses a fixed global threshold T to compare each pixel in the grayscale image with the global threshold T. When the grayscale value is less than the preset threshold T, the image is set to white and all three channels are set to 255; otherwise, it is set to black and all three channels are set to 0. The binarization formula is as follows: ; Among them, B( x , y ) is the binary image with coordinates ( x , y ), Gray ( x , y ) is the grayscale image with coordinates ( x , y ), T represents the custom threshold.

[0028] The feature vector of the key point in step S312 is a curve whose outline is composed of points connected by the same gray value, which represents the shape and boundary of the object, or detects the descriptor of the key point of the current frame according to different retrieval modes and approximation methods. All the key points detected in the current frame generate an 11-dimensional feature vector ; The optimal feature vector V It is composed of the following elements: V ={x coordinate of the key point center, y coordinate of the key point center, minimum enclosing rectangle border length, minimum enclosing rectangle border width, number of all points on the key point contour, contour hierarchy information, average pixel inside the key point contour, average pixel around the feature point contour extending 10 pixels outward}; The hierarchical information of the contour is a vector of length 4, which respectively represents the index of the next, previous, first child node and parent node of the current contour.

[0029] The similarity score expression in step S314 is as follows: ; in, V i It is i The feature vector of the key point, D P，N Indicates that the template dictionary is V i The feature vector of the most recent sample, α , β , δ is the weight coefficient, satisfying ; KL(·) is the KL divergence, which is used to measure the difference between two probability distributions. The expression for measuring the difference between two probability distributions is as follows: ; in, P and Q are two discrete probability distributions, x is a possible value, log is the natural logarithm; the expression that measures the difference between two probability distributions means to calculate the probability from the distribution Q To approximate the distribution P The amount of information required, i.e. the loss of information entropy; KL The larger the divergence, the more dissimilar the two distributions are; ED(•) is the Euclidean distance, which is used to measure the spatial distance between two vectors. The Euclidean distance expression is as follows: ; in, u and v There are two n dimensional vector, u i and v i They are in the i Coordinates in dimensions; the meaning of the Euclidean distance expression is to calculate the straight-line distance between two vectors in Euclidean space. The smaller the Euclidean distance, the closer the two vectors are. CS(·) is the cosine similarity, which is used to measure the directional similarity of two vectors; the cosine similarity expression is as follows: ; in, u and v There are two n dimensional vector, u i and v i They are in the i The cosine similarity expression means to calculate the cosine value of the angle between two vectors, that is, the inner product between their projections on the unit circle; the closer the cosine similarity is to 1, the more parallel the two vectors are; the closer the cosine similarity is to -1, the more opposite the two vectors are; the cosine similarity is 0, which means the two vectors are orthogonal.

[0030] The specific process of using key point matching to track the new position frame by frame in step S32 is as follows: S321: Create a corresponding tracker list based on the detected key points To store the displacement time series of the frames before and after the key points, Indicates tFrame n The tracker of key points stores the nth key point t The coordinate sequence of the frame, Indicates n The key point is t Frame x , y Axis coordinates; S322: For each tracker of each frame, a preset algorithm is used to match the optimal displacement between the key points of the previous and next frames; A similarity metric is proposed through the coordinates of the tracker of the previous frame and the coordinates of the key points of the current frame and their corresponding types of similarity metrics to match the key points of the previous and next frames. The expression is as follows: ; in, T i The previous frame i The coordinates of the trackers, K j The current frame j The coordinates of the key points, y i and y j are the corresponding key point types, s is the key point scale, ED is the Euclidean distance, which is used to measure the spatial distance between two coordinates. kd is the Kronecker function, which is used to determine whether two types are the same. If they are the same, it is 1, otherwise it is 0. The expression comprehensively measures the spatial proximity and semantic consistency of the key points in the previous and next frames to obtain a similarity value. The higher the similarity value, the more matched it is, and the lower the similarity value, the less matched it is. S323: When n Trackers are matched successfully, and the coordinate sequence of the key points that are currently matched successfully is appended Tracking in the tracker, the formula is: ; S324: Repeat S322 and S323 until all trackers of the previous frame are matched or all key points of the current frame are matched; S325: For a tracker that does not track displacement or loses key points due to occlusion or blur in the image, a Kalman filter is used to predict the most likely position of the lost key point, and the most similar candidate key point is searched near the position. When the most similar candidate key point is found, the tracker is updated, otherwise the prediction state is maintained until a match is found or the maximum allowed number of lost frames is exceeded; Alternatively, the optical flow method is used to estimate the direction and speed of the lost key point, and the most similar candidate key point is found along the motion trajectory in that direction. When the most similar candidate key point is found, the tracker is updated, otherwise the motion state is maintained until a match is found or the maximum allowed number of lost frames is exceeded; Alternatively, a neural network model that can identify and recover lost key points is trained through deep learning methods, and the model is used to generate the most likely candidate key points in the current frame. If the generation is successful, the tracker is updated, otherwise the generation state is maintained until the generation matches or the maximum allowed number of lost frames is exceeded.

[0031] The real-time vibration monitoring algorithm of step S3 includes a Fourier transform algorithm, a wavelet transform algorithm, and a power spectrum density estimation algorithm; Among them, the process of Fourier transform solving the frequency domain signal is as follows: Since any periodic function can be expressed as a linear combination of sine waves and cosine waves, namely the Fourier series, its mathematical expression is as follows: ; in, is the time domain signal, is the fundamental frequency, , , are the Fourier coefficients; when the period of a periodic function approaches infinity, the Fourier series becomes the Fourier transform.

[0032] The Fourier transform signal processing technology decomposes the signal into multiple sinusoidal signals. If the signal satisfies the Dirichlet condition, combined with the Euler formula, the Fourier transform can be performed; the expressions of the Euler formula and the Fourier transform are as follows: ; ; In the formula, j is an imaginary unit, is the angle, represents the argument of the complex number on the complex plane, that is, the angle between the complex number and the positive real axis. is the Fourier series in the frequency domain, is the fundamental frequency, is the time domain signal, represents the Euler number, which is an irrational constant. ≈2.71828.......

[0033] The calculation process of the vibration amplitude in step S4 is as follows: S41: Select a feature point and measure the length l of the feature point in the image in pixels; S42: Calculate the displacement d of the feature point in the horizontal direction and the vertical direction by comparing images of different time series; S43: Calculate the vibration amplitude by the displacement d of the feature point in the horizontal and vertical directions: Using the principle of similar triangles, the actual size s of the image in millimeters; the resolution r of the image in pixels / mm, calculate the actual displacement of the feature point in millimeters, that is, the vibration amplitude, by the formula: A = d*s / r; Finally, the vibration amplitude A is obtained, which is the maximum deviation distance of the target object in the actual space; Alternatively, the horizontal distance d between the camera and the target and the angle θ between the line connecting the camera and the target and the horizontal direction can be obtained through a laser or hardware distance measuring device; the displacement A of the target can be calculated using the relationship between trigonometric functions, that is: A=d*tan(θ); Finally, the vibration amplitude A is obtained, which is the maximum deviation distance of the target object in the actual space, or the actual vibration amplitude of the object is obtained through the displacement sensor; Alternatively, a three-axis acceleration sensor or a three-dimensional laser scanner is used to measure a certain position of the target object, and the vibration information of the position on the x-axis, y-axis and z-axis is obtained, and the data processing module is used to decompose and synthesize the information to obtain the distribution and changes of the vibration amplitude and frequency of the position in different directions.

[0034] A visualized real-time vibration monitoring system comprises an image acquisition device, a data processing module and a visualization module; the image acquisition device is used to acquire image information of a specified position of a device to be tested; the data processing module is used to analyze data containing vibration information and calculate vibration frequency and amplitude; the visualization module is used to generate and display a real-time visualized report based on the output of the data processing module; wherein the visualized report is in the form of charts, curves, images, videos and sounds.

[0035] The present invention collects vibration information of the target object and converts the vibration information into a digital signal. For example, a lens is used to shoot the target object, a video or picture sequence of the target object is obtained, and the video or picture sequence is converted into a digital signal. Alternatively, a laser, sound wave, infrared ray, etc. is used to scan the target object, a reflection signal of the target object is obtained, and the reflection signal is converted into a digital signal. Alternatively, an acceleration sensor, a displacement sensor, etc. are used to measure the target object, a motion signal of the target object is obtained, and the motion signal is converted into a digital signal. In addition, multiple cameras or sensors can be used to shoot or measure different positions of the target object to obtain vibration information at different positions. For the vibration conditions of the same position on different axes, a three-axis acceleration sensor or a three-dimensional laser scanner can be used to measure the target object, obtain the vibration information of the position on the x-axis, y-axis and z-axis, and a comprehensive analysis is performed through a data processing module to obtain the overall vibration characteristics and local vibration anomalies of the target object.

[0036] The present invention analyzes the data containing vibration information through a preset algorithm to calculate the vibration frequency and amplitude. The data containing vibration information is subjected to frequency domain analysis through Fourier transform to obtain a vibration spectrum, and the main vibration frequency and amplitude are extracted therefrom. The data containing vibration information is subjected to multi-scale analysis through wavelet transform to obtain vibration characteristics at different scales, and the main vibration frequency and amplitude are extracted therefrom. The data containing vibration information is subjected to power analysis through power spectrum density estimation to obtain a vibration power spectrum, and the main vibration frequency and amplitude are extracted therefrom. Decomposition and synthesis are performed through a data processing module to obtain the distribution and change of the vibration amplitude and frequency of the target object in different directions.

[0037] Based on the calculated vibration frequency and vibration amplitude, a real-time visual report is generated and displayed to the user. The user can view the vibration status of the target object in real time through the graphical interface. Use waveforms, bar charts, pie charts, etc. to display the distribution of vibration amplitude and frequency, and use color, size, shape, etc. to identify abnormal or dangerous areas. Or use curves, scatter plots, heat maps, etc. to display the changes in vibration amplitude and frequency over time or space, and use sound, text, icons, etc. to remind users to pay attention or take measures. Or use three-dimensional models, virtual reality, augmented reality, etc. to display the real-time status of the target object, and use animation, special effects, interaction, etc. to enhance the user experience. In addition, the vibration conditions of different positions or different axes can be displayed in the form of charts, etc., and the sorting, filtering, comparison and other functions can be used to help users analyze vibration data.

[0038] The present invention can generate different visualization reports in different ways to meet different user needs and scenarios. Figure 2 A visual report diagram: Figure 2 The lower right side is a visual report showing the distribution of the target object's vibration amplitude and frequency on the same axis in the form of a line graph. It can be seen that the target object's vibration intensity at different frequencies, as well as the range and center value of the vibration frequency; it can also be seen that the target object's vibration conditions on different axes require special attention or measures to be taken according to the actual situation.

[0039] Figure 2 The upper right side shows a visual report of the target object's vibration amplitude and frequency changes on different axes in the form of a curve chart. It can be seen that the vibration change trend of the target object at different time points, and whether there is periodicity or mutation.

[0040] Figure 2 The left is a visual report showing the real-time status of the target object in the form of a picture. The picture shows the vibration frequency and amplitude of the target object on the x-axis and y-axis in real time, which shows the overall shape and detailed features of the target object, as well as the relationship between the target object and the surrounding environment. You can directly observe the visual report of the vibration conditions of the target object at different positions or different axes, and you can see the vibration parameters of the target object at each position or on each axis, and analyze the vibration data through sorting, filtering, comparison and other functions.

[0041] In summary, the visualized real-time vibration monitoring algorithm and system provided by the present invention deploy image acquisition devices at different positions of the device to be tested, and obtain image information of the position to be tested of the device to be tested through the image acquisition devices; obtain vibration information of the position to be tested by obtaining the image information of the position to be tested of the device to be tested, and convert the analog signal of the vibration information into a digital signal; perform data analysis on the digital signal of the position to be tested based on the visualized real-time vibration monitoring algorithm; obtain the vibration frequency and vibration amplitude of the position to be tested of the device to be tested based on the data analysis calculation; generate a real-time visualized report of different positions of the device to be tested using the obtained vibration frequency and vibration amplitude of the position to be tested of the device to be tested.

[0042] Avoiding the installation of sensors on the equipment to be tested reduces space occupation and will not affect the normal operation of the equipment. At the same time, it avoids the vibration monitoring results being affected by factors such as ambient temperature, humidity, and electromagnetic interference, thereby improving the accuracy of the monitoring results. It can cover all parts where abnormal vibrations may occur, and realize real-time monitoring of the vibration of each monitoring point of the equipment to be tested, without problems such as data delay, loss, and interference. The data processing center does not require a large amount of storage space and computing resources, and at the same time solves the technical problems of low detection efficiency, waste of manpower and material resources, and large errors in handheld or portable vibration detection instruments.

Claims

1. A visual real-time vibration monitoring algorithm, characterized in that: The following steps are involved: S1: deploying image acquisition devices at different positions of the device to be tested, and acquiring image information of the position to be tested of the device to be tested through the image acquisition devices; S2: obtaining vibration information of the position to be tested by acquiring image information of the position to be tested of the device to be tested, and converting an analog signal of the vibration information into a digital signal; S3: Data analysis of digital signals at the measured position based on real-time vibration monitoring algorithm based on visualization; S4: Obtaining the vibration frequency and vibration amplitude of the position to be tested of the device to be tested based on data analysis and calculation; S5: Generate a real-time visual report of different positions of the device under test using the obtained vibration frequency and vibration amplitude of the device under test at the position under test.

2. A visual real-time vibration monitoring algorithm according to claim 1, characterized in that: Step S1 includes the following specific processes: S11: Mark the designated position of the device to be tested, that is, mark the key point; S12: The image information of key points of the device to be tested is collected by an image acquisition device as a basis for vibration data of the device to be tested.

3. A visual real-time vibration monitoring algorithm according to claim 1, characterized in that: Step S3 includes the following specific processes: S31: Detect key points of the device to be tested using image information of the key points; S32: Use keypoint matching to track the new position frame by frame; S33: Build a vibration monitoring model and perform data analysis on key points.

4. A visual real-time vibration monitoring algorithm according to claim 3, characterized in that: Step S31 The specific process includes the following: S311: graying the image by weighted average method. According to the different sensitivity of human eyes to different colors, the green channel has the highest weight, the red channel has the second highest weight, and the blue channel has the lowest weight. Threshold segmentation is performed by binarization method. S312: Detect and extract feature vectors of key points from the binary image; S313: Establishing a PN template dictionary; S314: Determine the category of the corresponding key point by calculating and comparing the similarity score between the PN template dictionary and the feature vector.

5. A visual real-time vibration monitoring algorithm according to claim 4, characterized in that: The formula of the weighted average method in step S311 is: ； in, Represents the grayscale image after grayscale conversion. R , G , B Represents the red, green, and blue channels of a color image respectively; The binarization method uses a fixed global threshold T to compare each pixel in the grayscale image with the global threshold T. When the grayscale value is less than the preset threshold T, the image is set to white and all three channels are set to 255; otherwise, it is set to black and all three channels are set to 0. The binarization formula is as follows: ； Among them, B( x , y ) is the binary image with coordinates ( x , y ) pixel value, Gray( x , y ) is the grayscale image with coordinates ( x , y ), T represents the custom threshold.

6. A visual real-time vibration monitoring algorithm according to claim 4, characterized in that: The feature vector of the key point in step S312 is a curve whose outline is composed of points connected by the same gray value, which represents the shape and boundary of the object, or detects the descriptor of the key point of the current frame according to different retrieval modes and approximation methods. All the key points detected in the current frame generate an 11-dimensional feature vector ; The optimal feature vector V It is composed of the following elements: V ={x coordinate of the key point center, y coordinate of the key point center, minimum enclosing rectangle border length, minimum enclosing rectangle border width, number of all points on the key point contour, contour hierarchy information, average pixel inside the key point contour, average pixel around the feature point contour extending 10 pixels outward}; The hierarchical information of the contour is a vector of length 4, which respectively represents the indexes of the next, previous, first child node and parent node of the current contour.

7. A visual real-time vibration monitoring algorithm according to claim 4, characterized in that: The similarity score expression in step S314 is as follows: ； in, V i It is i The feature vector of the key point, D P，N Indicates that the template dictionary is V i The feature vector of the most recent sample, α , β , δ is the weight coefficient, satisfying ; KL(·) is the KL divergence, which is used to measure the difference between two probability distributions. The expression for measuring the difference between two probability distributions is as follows: ； in, P and Q are two discrete probability distributions, x is a possible value, log is the natural logarithm; the expression that measures the difference between two probability distributions means to calculate the probability from the distribution Q To approximate the distribution P The amount of information required, i.e. the loss of information entropy; KL The larger the divergence, the more dissimilar the two distributions are; ED(•) is the Euclidean distance, which is used to measure the spatial distance between two vectors. The Euclidean distance expression is as follows: ； in, u and v There are two n dimensional vector, u i and v i They are in the i Coordinates in dimensions; the meaning of the Euclidean distance expression is to calculate the straight-line distance between two vectors in Euclidean space. The smaller the Euclidean distance, the closer the two vectors are. CS(·) is the cosine similarity, which is used to measure the directional similarity of two vectors; the cosine similarity expression is as follows: ； in, u and v There are two n dimensional vector, u i and v i They are in the i The cosine similarity expression means to calculate the cosine value of the angle between two vectors, that is, the inner product between their projections on the unit circle; the closer the cosine similarity is to 1, the more parallel the two vectors are; the closer the cosine similarity is to -1, the more opposite the two vectors are; the cosine similarity is 0, which means the two vectors are orthogonal.

8. A visual real-time vibration monitoring algorithm according to claim 1, characterized in that: The specific process of using key point matching to track the new position frame by frame in step S32 is as follows: S321: Create a corresponding tracker list based on the detected key points To store the displacement time series of the frames before and after the key points, Indicates t Frame n The tracker of key points stores the nth key point t The coordinate sequence of the frame, Indicates n The key point is t Frame x , y Axis coordinates; S322: For each tracker of each frame, a preset algorithm is used to match the optimal displacement between the key points of the previous and next frames; The key points of the previous and next frames are matched by the tracker coordinates of the previous frame and the coordinates of the key points of the current frame and the similarity measurement of the corresponding type. The expression is as follows: ； in, T i The previous frame i The coordinates of the trackers, K j The current frame j The coordinates of the key points, y i and y j are the corresponding key point types, s is the key point scale, ED is the Euclidean distance, which is used to measure the spatial distance between two coordinates. kd is the Kronecker function, which is used to determine whether two types are the same. If they are the same, it is 1, otherwise it is 0. The expression comprehensively measures the spatial proximity and semantic consistency of the key points in the previous and next frames to obtain a similarity value. The higher the similarity value, the more matched it is, and the lower the similarity value, the less matched it is. S323: When n Trackers are matched successfully, and the coordinate sequence of the key points that are currently matched successfully is appended Tracking in the tracker, the formula is: ； S324: Repeat S322 and S323 until all trackers of the previous frame are matched or all key points of the current frame are matched; S325: For a tracker that does not track displacement or loses key points due to occlusion or blur in the image, a Kalman filter is used to predict the most likely position of the lost key point, and the most similar candidate key point is searched near the position. When the most similar candidate key point is found, the tracker is updated, otherwise the prediction state is maintained until a match is found or the maximum allowed number of lost frames is exceeded; Alternatively, the optical flow method is used to estimate the direction and speed of the lost key point, and the most similar candidate key point is found along the motion trajectory in that direction. When the most similar candidate key point is found, the tracker is updated, otherwise the motion state is maintained until a match is found or the maximum allowed number of lost frames is exceeded; Alternatively, a neural network model that can identify and recover lost key points is trained through deep learning methods, and the model is used to generate the most likely candidate key points in the current frame. If the generation is successful, the tracker is updated, otherwise the generation state is maintained until the generation matches or the maximum allowed number of lost frames is exceeded.

9. A visual real-time vibration monitoring algorithm according to claim 1, characterized in that: The real-time vibration monitoring algorithm of step S3 includes a Fourier transform algorithm, a wavelet transform algorithm, and a power spectrum density estimation algorithm; The calculation process of the vibration amplitude in step S4 is as follows: S41: Select a feature point and measure the length l of the feature point in the image in pixels; S42: Calculate the displacement d of the feature point in the horizontal direction and the vertical direction by comparing images of different time series; S43: Calculate the vibration amplitude by the displacement d of the feature point in the horizontal and vertical directions: Using the principle of similar triangles, the actual size s of the image in millimeters; The resolution r of the image in pixels / mm is used to calculate the actual displacement of the feature point, i.e. the vibration amplitude, in millimeters. The formula is: A = d*s / r; Finally, the vibration amplitude A is obtained, which is the maximum deviation distance of the target object in the actual space; Alternatively, the horizontal distance d between the camera and the target and the angle θ between the line connecting the camera and the target and the horizontal direction can be obtained through a laser or hardware distance measuring device; the displacement A of the target can be calculated using the relationship between trigonometric functions, that is: A=d*tan(θ); Finally, the vibration amplitude A is obtained, which is the maximum deviation distance of the target object in the actual space; Or obtain the actual vibration amplitude of the object through a displacement sensor; Or use a three-axis acceleration sensor or a three-dimensional laser scanner to measure a certain position of the target object and obtain the position x axis, y Axis and z The vibration information on the axis is decomposed and synthesized through the data processing module to obtain the distribution and changes of the vibration amplitude and frequency in different directions at that position.

10. A visual real-time vibration monitoring system, characterized in that: include: Image acquisition device, data processing module and visualization module; The image acquisition device is used to acquire image information of a specified position of the device to be tested; The data processing module is used to analyze the data containing vibration information and calculate the vibration frequency and amplitude; The visualization module is used to generate and display real-time visualization reports based on the output of the data processing module; wherein the visualization reports are in the form of charts, curves, images, videos and sounds.

Images