A vibration monitoring method and device based on a mobile terminal and image processing

Abstract

The present invention discloses a vibration monitoring method and device based on a mobile terminal and image processing, belonging to the technical field of vibration equipment monitoring. A vibration monitoring method based on a mobile terminal and image processing according to the present invention constructs an image similarity and spectrum correction model to estimate the rotation angle between a to-be-detected image and a template image; based on the Hamming distance, corresponding matching is performed on feature points in the to-be-detected image and the template image, and data fitting is performed on the successfully matched feature point pairs to obtain the position change between the to-be-detected image and the template image, so as to realize vibration monitoring of a rotating component. The solution is scientific and reasonable, has universal applicability, solves the problem of difficult sensor layout and wiring in the prior art, improves the anti-interference ability, dynamic response ability, stability and reliability of the sensor while ensuring continuous and effective vibration information can be obtained, and reduces maintenance and calibration costs.

Description

Technical Field

[0001] The present invention relates to a vibration monitoring method and device based on a mobile terminal and image processing, and belongs to the technical field of vibration equipment monitoring. Background Art

[0002] Vibration monitoring technology is widely used in industrial production processes, such as the condition monitoring of some important mechanical equipment, such as generators, motors, coal mills, wind turbines, etc. Vibration is one of the important parameters reflecting the operating conditions of rotating machinery. Real-time and accurate vibration monitoring not only helps to timely grasp the operating state of rotating machinery, but also provides key information for equipment maintenance and fault diagnosis. According to statistics, more than 70% of the fault conditions of rotating machinery can be diagnosed using vibration characteristics. Existing vibration detection sensors include piezoelectric acceleration sensors, eddy current sensors, laser Doppler instruments, electromagnetic velocity sensors, etc. Although these sensors have achieved certain effects in practical applications, they all have certain limitations in terms of accuracy, range, applicability, etc.

[0003] The piezoelectric acceleration sensor is restricted by the installation method and cannot achieve non-contact measurement of rotating components. It is prone to wear during long-term use. In addition, the object to be measured needs to maintain mechanical contact with it, which correspondingly restricts the installation layout of the measurement object. At the same time, it also makes the object under test bear additional mass, which may have an adverse impact on its dynamic characteristics and the final measurement result. In particular, for lightweight mechanical structures and fine and small devices, such as microelectromechanical systems, rotor bearings and other devices, the impact of such devices is particularly serious.

[0004] Non-contact monitoring methods such as laser Doppler instruments and electronic speckle interferometry overcome the defects of contact measurement technology to a certain extent, but such devices are expensive and the system is complex. They often require complex optical paths and auxiliary devices.

[0005] Eddy current displacement sensors and electromagnetic velocity sensors are restricted by their measurement principles and have strict requirements for the environment. In particular, they are sensitive to magnetic interference and cannot directly measure non-metallic or alloy materials. In addition, such devices generally need to be fixedly installed around the rotating component under test. Although it can ensure the accuracy and reliability of the measurement, it will cause additional assembly costs, poor portability, and cannot perform far-field measurement. The on-site wiring of these devices is also not conducive to wide application in industrial occasions.

[0006] In recent years, many scholars at home and abroad have begun to study image-based non-contact vibration monitoring methods. The research idea is to preset marks on the object to be measured, and use the camera to track the position change information of the mark to deduce the kinematic parameters. Existing visual measurement systems generally require the use of expensive high-speed cameras and set extremely short exposure times to obtain clear images and reduce the impact of motion blur on image quality. This type of method has high requirements on image resolution and frame rate, and is easily affected by adverse factors such as target occlusion and noise. Given that the cameras carried by mobile terminals generally have low frame rates and resolutions, it is difficult for existing visual monitoring methods to overcome the adverse effects of motion blur and cannot be applied to portable devices such as mobile terminals. Summary of the invention

[0007] In view of the defects of the prior art, the first object of the present invention is to provide a method for screening several images of rotating parts taken by a mobile terminal: selecting one image from the several images as a template image, and selecting one or more images other than the template image as images to be tested; then constructing an image similarity and spectrum correction model, and using the normalized cross-correlation coefficient and the energy center of gravity correction method to estimate the rotation angle between the image to be tested and the template image; then rotating and resetting the image to be tested according to the rotation angle to form a reset image, so as to reduce the adverse effect of rotation on vibration measurement; and then constructing a nonlinear diffusion filtering model, extracting feature points from the reset image and the template image, and forming a feature point descriptor; thereby, the vibration measurement can be based on the image similarity and spectrum correction model. According to the feature descriptor, the feature points in the image to be tested and the template image are matched accordingly with the Hamming distance as the benchmark, and the data fitting is performed on the successfully matched feature point pairs to obtain the position change of the image to be tested compared with the template image; finally, according to the calibration coefficient between the image pixel and the actual displacement, the position change is converted into the actual vibration displacement of the rotating part to realize the vibration monitoring of the rotating part. The scheme is scientific and reasonable, and has universal applicability. It solves the problem of the difficulty in the existing sensor point layout and wiring. While ensuring the ability to obtain continuous and effective vibration information, it improves the sensor's anti-interference ability, dynamic response ability, stability and reliability, and reduces the maintenance and calibration costs of the vibration monitoring method based on mobile terminals and image processing.

[0008] In view of the defects of the prior art, the second purpose of the present invention is to provide a non-contact vibration monitoring method, which will not affect the dynamic performance of the object being measured, can meet the needs of long-distance vibration monitoring anytime and anywhere, and bring great convenience to industrial site operations; for image sequences with blur, brightness and rotation changes, it has higher measurement performance, which is superior to the vibration monitoring device based on mobile terminal and image processing of the existing visual measurement device.

[0009] To achieve one of the above purposes, the first technical solution of the present invention is:

[0010] A vibration monitoring method based on a mobile terminal and image processing

[0011] The method includes the following steps:

[0012] Step 1: Obtain a number of images of a rotating component taken by a mobile terminal;

[0013] The images are used to record the pose states of the rotating component at different times;

[0014] Step 2: Screen the number of images in Step 1;

[0015] The screening process is as follows:

[0016] Select one image from the number of images as a template image,

[0017] Select one or more images other than the template image as test images;

[0018] Step 3: Construct an image similarity and spectrum correction model;

[0019] The image similarity and spectrum correction model uses the normalized cross-correlation coefficient and the energy center of gravity correction method to estimate the rotation angle between the test image and the template image in Step 2;

[0020] Step 4: Rotate and reset the test image according to the rotation angle calculated in Step 3 to form a reset image, so as to reduce the adverse effect of rotation on vibration measurement;

[0021] Step 5: Construct a non-linear diffusion filtering model, filter the reset image in Step 4 and the template image in Step 2 to obtain a non-linear scale space;

[0022] After that, extract the feature points of the reset image and the template image in the non-linear scale space, and generate feature point descriptors for the extracted feature points according to their scale and direction information;

[0023] Step 6: According to the feature descriptors in Step 5, use the Hamming distance as a benchmark to perform corresponding matching on the feature points in the test image and the template image, and perform data fitting on the successfully matched feature point pairs to obtain the position change of the test image relative to the template image;

[0024] Step 7: According to the calibration coefficient between the image pixels and the actual displacement, convert the position change in Step 6 into the actual vibration displacement of the rotating component to realize the vibration monitoring of the rotating component.

[0025] Through continuous exploration and experiments, the present invention screens a number of images of a rotating component taken by a mobile terminal:

[0026] Select one image from several images as the template image, and select one or more images other than the template image as the images to be measured; then construct an image similarity and spectrum correction model, and use the normalized cross-correlation coefficient and the energy centroid correction method to estimate the rotation angle between the images to be measured and the template image; then, according to the rotation angle, rotate and reset the images to be measured to form reset images, so as to reduce the adverse impact of rotation on vibration measurement; furthermore, construct a non-linear diffusion filtering model, extract feature points from the reset images and the template image to obtain feature point descriptors; thus, according to the feature descriptors, using the Hamming distance as the benchmark, perform corresponding matching on the feature points in the images to be measured and the template image, and perform data fitting on the successfully matched feature point pairs to obtain the position change of the images to be measured compared with the template image; finally, according to the calibration coefficient between the image pixels and the actual displacement, convert the position change into the actual vibration displacement of the rotating component to realize the vibration monitoring of the rotating component. The scheme is scientific and reasonable, has general applicability, solves the problem of difficult sensor layout and wiring in the prior art, improves the anti-interference ability, dynamic response ability, stability and reliability of the sensor while ensuring continuous and effective vibration information can be obtained, and reduces the maintenance and calibration costs.

[0027] Furthermore, as a non-contact vibration monitoring means, the vibration monitoring method of the present invention will not affect the dynamic performance of the object to be measured, can meet the vibration remote monitoring requirements at any time and place, and brings great convenience to industrial on-site operations.

[0028] Still further, the present invention uses an image similarity and centroid correction model to calculate the rotation angle between the template image and the images to be measured, so that the images to be measured are rotated and reset, which can reduce the influence of image rotation on vibration measurement.

[0029] Even further, the present invention uses the Hamming distance to perform image matching and target monitoring on local AKAZE features, can perform vibration monitoring on image sequences affected by adverse effects such as blurring, brightness and rotation changes, and furthermore, the present invention has high measurement performance and is superior to existing vision measurement methods.

[0030] As a preferred technical measure:

[0031] In step 1, the method for image acquisition is as follows:

[0032] Use the camera of the mobile terminal to shoot the vibration videos of the rotating component in different working states, store them in the data storage module, and then process the vibration videos to obtain several images;

[0033] During the shooting process, the shooting frame rate of the camera remains unchanged, and the optical axis of the camera is perpendicular to the cross-section of the rotating component;

[0034] The rotating component is an engine rotating shaft, a motor rotor, or a steam turbine blade.

[0035] As a preferred technical measure:

[0036] In step 3, the method for constructing the image similarity and spectrum correction model is as follows:

[0037] S31: Use the normalized cross-correlation algorithm to calculate the normalized cross-correlation coefficient between each image in the images to be measured and the template image, and convert the images to be measured into a one-dimensional signal S(k) representing image similarity;

[0038] The calculation formula for the normalized cross-correlation coefficient is as follows:

[0039]

[0040]

[0041]

[0042]

[0043]

[0044] Among them, f(x i , y j ) is the template image, g(x′ i , y′ j ) is the image to be measured, and m×n is the resolution;

[0045] S32: Use the energy centroid correction method to perform spectrum correction on the one-dimensional signal S(k) representing image similarity in S31, and extract the high-precision rotation frequency f;

[0046] According to the centroid characteristics, the calculation formula for the rotation frequency f after the similarity signal correction is as follows:

[0047]

[0048] In the formula, G i is the amplitude of the i-th peak spectral line of the power spectrum of the one-dimensional signal S(k) representing image similarity,

[0049] G k corresponds to the main frequency component corresponding to the maximum amplitude of the power spectrum within the main lobe,

[0050] k is the spectral line number corresponding to the point with the maximum amplitude,

[0051] N is the length of the similarity signal,

[0052] f s is the sampling frequency of the camera,

[0053] S33: Obtain the rotation angle of the image to be measured relative to the template image according to the relationship between the rotation frequency f in S32 and the acquisition time T of the image to be measured. The calculation formula for the rotation angle is as follows:

[0054] θ = f × T × 360°.

[0055] As an optimal technical measure:

[0056] In step 5, the construction method of the non-linear diffusion filtering model is as follows:

[0057] S51: Introduce a scale parameter, filter the reset image and the template image using the non-linear diffusion filtering equation, and construct a scale space to reduce the amplitude change of image pixels;

[0058] The calculation formula of the non-linear diffusion filtering equation is as follows:

[0059]

[0060] where (x, y) are the pixel coordinates of the image point,

[0061] div represents divergence,

[0062] is the gradient,

[0063] L represents the image brightness,

[0064] t is the scale parameter,

[0065] c is the conduction diffusion function;

[0066] The calculation formula of the conduction diffusion function is as follows:

[0067]

[0068]

[0069] where, represents Gaussian smoothing operation, δ represents the standard deviation, and λ represents the image contrast factor;

[0070] S52: Establish a pyramid image model according to the scale space in S51;

[0071] The establishment process of the pyramid image model is as follows:

[0072] Determine the total number of layers O of the pyramid and the number of blurred images S in each layer;

[0073] For the s-th image in the o-th layer of the pyramid, determine the corresponding blurred scale parameter;

[0074] The calculation formula of the fuzzy scale parameter is as follows:

[0075] σ i (o,s)=σ02 o+s / S ,o∈[0,1,…,O - 1],s∈[0,1,…,S - 1],i∈[0,1,…,M - 1](10)

[0076] Among them, σ0 is the reference value of the scale space parameter, and M = O×S is the total number of all images included in the scale space;

[0077] Map the fuzzy scale parameter to time units, and its mapping formula is as follows:

[0078]

[0079] S53: Use the non - maximum suppression method to calculate the Hessian matrix value of each pixel point in the pyramid image model in S52, and perform scale normalization processing to obtain the normalized Hessian matrix value;

[0080] S54: Compare the normalized Hessian matrix value in S53 with 26 pixel points in the same layer and the upper and lower adjacent layers, find the maximum value of the Hessian matrix, and select it as the feature point;

[0081] S55: Taking the feature point in S54 as the center, with a statistical range of radius 6σ, take the first - order differentials Lx and Ly of the feature point neighborhood for Gaussian weighting operation; and rotate around the center with a 60° sector area, calculate the vector sum within this area, and take the longest direction as the main direction of the feature point;

[0082] S56: Use the local differential descriptor M - LDB to divide the feature point neighborhood in S55 into several sub - grids, and resample at intervals of scale σ in the sub - grids to obtain discrete points;

[0083] S57: Calculate the mean value of the brightness value, horizontal and vertical direction derivatives of the discrete points in S56, and let the mean values of the two derivatives form a 3 - bit - length feature descriptor.

[0084] As a preferred technical measure:

[0085] In step 6, the matching method is as follows:

[0086] Use the Hamming distance to match the feature descriptors with the same length, find the pixel coordinates of the same spatial point in the image to be measured and the template image respectively, obtain the feature matching point pairs, and perform geometric transformation between the template image and the image to be measured;

[0087] The geometric transformation is a full affine transformation, and its calculation formula is as follows:

[0088]

[0089] Among them, (x1, y1), (x2, y2), …, (x q , y q )(q ≥ 3) are the pixel coordinates of the feature points of the reference image f(x i , y j ), and (x′1, y′1), (x′2, y′2), …, (x′ q , y′ q ) are the pixel coordinates in the image to be measured g(x′ i , y′ j ) that match them.

[0090] a, b, c, and d are parameters obtained by data fitting.

[0091] t x and t y are the displacement amounts of the rotating component on the x-axis and y-axis respectively, representing the vibration of the rotating component.

[0092] Perform data fitting on the feature matching point pairs to solve for t x and t y .

[0093] As a preferred technical measure:

[0094] In step 7, the actual vibration displacement is obtained by converting t x and t y into the vibration displacement in the actual space according to the calibration coefficient K between the pixel position and the actual displacement. The calculation formula is as follows:

[0095]

[0096] To achieve one of the above purposes, the second technical solution of the present invention is:

[0097] A vibration monitoring device based on a mobile terminal and image processing,

[0098] applying the above-mentioned vibration monitoring method based on a mobile terminal and image processing;

[0099] It has a mobile terminal device that can be arranged on one side of the rotating component to be measured;

[0100] The mobile terminal includes a camera, an image acquisition module, a data storage module, and an image and signal processing module;

[0101] The camera is used to photograph the moving surface of the rotating component;

[0102] The image acquisition module is used to set the frame rate, resolution, exposure time, and shooting angle of the camera according to the actual working conditions, control the camera to take pictures of the surface of the rotating component, and transmit the acquired image sequence to the data storage module;

[0103] The data storage module is used to store the image sequence;

[0104] The image and signal processing module is used to perform similarity calculation, energy center of gravity correction, local feature AKAZE detection and matching operations on the image sequence to obtain the actual vibration displacement of the rotating component.

[0105] The mobile terminal of the present invention is a new type of portable and non-contact vibration monitoring device, which can provide a new technical means for vibration monitoring and has broad application prospects. Compared with the existing vision monitoring system based on high-resolution high-speed cameras, it has the advantages of low cost, strong portability, and high measurement accuracy.

[0106] As a non-contact vibration monitoring means, the vibration monitoring device of the present invention will not affect the dynamic performance of the measured object, can meet the vibration remote monitoring requirements anytime and anywhere, and brings great convenience to industrial field operations.

[0107] The present invention uses an image similarity and spectrum correction model to calculate the rotation angle between the template image and the image to be measured, so that the image to be measured is rotated and reset, which can reduce the influence of image rotation on vibration measurement.

[0108] The present invention uses the Hamming distance to perform image matching and target monitoring on AKAZE features. For image sequences with blurring, brightness, and rotation changes, this device has high measurement performance, which is superior to existing vision measurement methods.

[0109] As a preferred technical measure:

[0110] The camera takes pictures at a fixed frame rate, and the setting of the frame rate is determined according to the vibration frequency range of the rotating component to prevent measurement failure caused by sampling aliasing; generally, a relatively high shooting frame rate should be set to obtain high measurement accuracy, and the frame rate should be not less than 30 frames per second;

[0111] The area occupied by the rotating component in the image is greater than or equal to 204,800 (640×480) pixels, so as to provide sufficient texture information for similarity calculation of the image sequence and local feature detection of the image.

[0112] As a preferred technical measure:

[0113] The exposure time of the camera is less than 1000 μs to avoid excessive motion blur, so as to ensure that sufficient image features can be extracted;

[0114] The light intensity is greater than 300 Lux to ensure better imaging quality;

[0115] The shooting angle of the camera is from -10 degrees to 10 degrees to ensure less image distortion and aberration;

[0116] The shooting distance of the camera is greater than or equal to 20 cm, and the specific value needs to be set according to the actual size of the rotating part and the camera resolution to ensure normal image focusing.

[0117] As a preferred technical measure:

[0118] The mobile terminal can be a drone equipped with a camera, a smart phone or a Raspberry Pi.

[0119] Compared with the prior art, the present invention has the following beneficial effects:

[0120] Through continuous exploration and experiments, the present invention screens several images of the rotating part taken by the mobile terminal:

[0121] Select one image from several images as the template image, and select one or more images other than the template image as the images to be measured; then construct an image similarity and spectrum correction model, and use the normalized cross-correlation coefficient and the energy center of gravity correction method to estimate the rotation angle between the images to be measured and the template image; then, according to the rotation angle, rotate and reset the images to be measured to form reset images, so as to reduce the adverse effects of rotation on vibration measurement; furthermore, construct a non-linear diffusion filtering model to filter the reset images and the template images in step 2, then extract feature points and obtain feature point descriptors; thus, according to the feature descriptors, with the Hamming distance as the benchmark, perform corresponding matching on the feature points in the images to be measured and the template images, and perform data fitting on the successfully matched feature point pairs to obtain the position change of the images to be measured compared with the template image; finally, according to the calibration coefficient between the image pixels and the actual displacement, convert the position change into the actual vibration displacement of the rotating part to realize the vibration monitoring of the rotating part. The solution is scientific and reasonable, has general applicability, solves the problem of difficult sensor layout and wiring in the prior art, and while ensuring continuous and effective vibration information can be obtained, improves the anti-interference ability, dynamic response ability, stability and reliability of the sensor, and reduces the maintenance and calibration costs.

[0122] Furthermore, as a non-contact vibration monitoring means, the vibration monitoring method of the present invention will not affect the dynamic performance of the measured object, can meet the vibration remote monitoring requirements anytime and anywhere, and brings great convenience to industrial field operations.

[0123] Furthermore, the present invention calculates the rotation angle between the template image and the image to be measured using an image similarity and spectrum correction model, so that the image to be measured is rotated and reset, which can reduce the influence of image rotation on vibration measurement.

[0124] Moreover, the present invention uses the Hamming distance to perform image matching and target monitoring on the local feature AKAZE. For image sequences with blur, brightness, and rotation changes, the present invention has higher measurement performance, superior to existing vision measurement methods.

[0125] Furthermore, as a non-contact vibration monitoring means, the vibration monitoring device of the present invention will not affect the dynamic performance of the object to be measured, can meet the needs of vibration remote monitoring anytime and anywhere, and brings great convenience to industrial on-site operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0126] Attached Figure 1 is a flowchart of the vibration monitoring method based on a mobile terminal and image processing according to the present invention;

[0127] Attached Figure 2 is a schematic structural diagram of the vibration monitoring device based on a mobile terminal and image processing according to the present invention.

[0128] Reference Signs:

[0129] 1. Rotating component; 2. Mobile terminal; 3. Image acquisition module; 4. Data storage module; 5. Image and signal processing module. DETAILED DESCRIPTION OF THE INVENTION

[0130] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0131] On the contrary, the present invention covers any alternatives, modifications, equivalent methods, and solutions made within the spirit and scope of the present invention as defined by the claims. Further, in order to enable the public to better understand the present invention, in the following detailed description of the present invention, some specific details are described in detail. Those skilled in the art can fully understand the present invention without the description of these details.

[0132] The first specific embodiment of the vibration monitoring method based on a mobile terminal and image processing according to the present invention:

[0133] A vibration monitoring method based on a mobile terminal and image processing,

[0134] comprises the following steps:

[0135] Step 1: Obtain a number of images of the rotating component captured by the mobile terminal;

[0136] The images are used to record the pose states of the rotating component at different moments;

[0137] Step 2: Screen the number of images in Step 1;

[0138] The screening process is as follows:

[0139] Select one image from the number of images as the template image,

[0140] Select one or more images other than the template image as the images to be tested;

[0141] Step 3: Construct an image similarity and spectral correction model;

[0142] The image similarity and spectral correction model uses the normalized cross-correlation coefficient and the energy center of gravity correction method to estimate the rotation angle between the images to be tested and the template image in Step 2

[0143] Step 4: Rotate and reset the images to be tested according to the rotation angle calculated in Step 3 to form reset images, so as to reduce the adverse effects of rotation on vibration measurement;

[0144] Step 5: Construct a nonlinear diffusion filtering model to filter the reset images in Step 4 and the template images in Step 2 to obtain a nonlinear scale space;

[0145] The filtering process is as follows:

[0146] Extract the feature points of the reset images and the template images, and generate feature point descriptors for the extracted feature points according to their scale and direction information;

[0147] Step 6: Based on the feature descriptors in Step 5, use the Hamming distance as the benchmark to perform corresponding matching on the feature points in the images to be tested and the template images, and perform data fitting on the successfully matched feature point pairs to obtain the position change of the images to be tested compared to the template images;

[0148] Step 7: According to the calibration coefficient between the image pixels and the actual displacement, convert the position change in Step 6 into the actual vibration displacement of the rotating component to realize the vibration monitoring of the rotating component.

[0149] The second specific embodiment of the vibration monitoring method based on the mobile terminal and image processing of the present invention:

[0150] A vibration monitoring method based on the mobile terminal and image processing specifically includes the following steps:

[0151] Step S1: Use the camera of the mobile terminal to capture vibration videos of the rotating component (such as the engine rotating shaft, steam turbine blade, etc.) in different working states, store them in the data storage module, and then call the image and signal processing module to process the images;

[0152] Step S2: Take the first captured image as the template image, and the remaining images as the images to be measured; Use image similarity calculation and energy centroid correction techniques to estimate the rotation angle between the images to be measured and the template image, and rotate the images to be measured to their original positions to reduce the adverse effects of rotation on subsequent feature extraction and matching;

[0153] Step S3: Introduce a scale parameter using a non-linear diffusion filter, perform non-linear diffusion filtering on the acquired images to construct a scale space; Extract the AKAZE features of each image, and for each extracted feature point, generate a feature point descriptor according to its scale, direction and other information;

[0154] Step S4: Based on the Hamming distance, match the AKAZE feature descriptors of the images to be measured and the template image; Perform data fitting on the successfully matched feature point pairs to solve the position change marked in the images to be measured; According to the calibration coefficient between the image pixels and the actual displacement, convert the position change in the marked image into the actual vibration displacement of the rotating component.

[0155] As Figure 1 shown, an optimal embodiment of the vibration monitoring method based on the mobile terminal and image processing of the present invention:

[0156] The vibration monitoring method based on the mobile terminal and image processing uses image and signal processing technologies to obtain vibration displacement, and detects the position change of the rotating component through image similarity evaluation, centroid correction method and local feature AKAZE algorithm, which specifically includes the following steps:

[0157] Step S1: Use the camera of the mobile terminal to capture vibration videos of the rotating component (such as the engine rotating shaft, steam turbine blade, etc.) in different working states, store them in the data storage module, and then call the image and signal processing module to process the images;

[0158] In the step S1, during the process of collecting the rotating component images, the shooting frame rate of the camera remains unchanged, and the optical axis of the camera is perpendicular to the cross-section of the rotor component;

[0159] Step S2: Take the first captured image as the template image, and use image similarity calculation and centroid correction techniques to estimate the rotation angle between the subsequent images and the template image, and rotate the images to be measured to their original positions.

[0160] In the step 2, it specifically includes:

[0161] S21: First, use the normalized cross - correlation algorithm to calculate the normalized cross - correlation coefficient between each image in the image sequence and the first image, and transform the image sequence into a one - dimensional signal S(k) representing image similarity; assume that the resolution of the template image f(x i ,y j ) and the image to be measured g(x′ i ,y′ j ) is m×n, and the formula for calculating the normalized cross - correlation coefficient between the image f(x i ,y j ) and the image g(x′ i ,y′ j ) is:

[0162]

[0163]

[0164]

[0165]

[0166]

[0167] S22: After obtaining the one - dimensional signal S(k) representing image similarity, use the energy - center correction method to perform spectrum correction on the reconstructed image similarity signal and extract the high - precision rotation frequency f. According to the center - of - gravity characteristic, the true frequency f after correcting the similarity signal is:

[0168]

[0169] where G i is the amplitude of the i - th peak spectral line of the power spectrum of the similarity signal S(k), G k corresponds to the main - frequency component corresponding to the maximum amplitude of the power spectrum within the main lobe, k is the spectral line number corresponding to the point with the maximum amplitude, N is the length of the similarity signal; f s is the sampling frequency of the camera. Finally, according to the relationship between the acquisition time T of the image to be measured and the rotation frequency, the rotation angle θ of the image to be measured relative to the template image is obtained as θ = f×T×360°;

[0170] Step S3: Use the local feature AKAZE detection algorithm to extract the local features of each image. For each feature point, generate a feature - point descriptor according to its scale, direction and other information for subsequent image matching.

[0171] Specifically, it includes:

[0172] S31: Use a non - linear diffusion filter to introduce a scale parameter, filter the original image, and construct a scale space. The non - linear diffusion filtering equation is:

[0173]

[0174] Among them, (x, y) represents the pixel coordinates of the image point, div represents the divergence, is the gradient, L represents the image brightness, t represents the scale parameter. c represents the conduction diffusion function and adopts the following formula:

[0175]

[0176]

[0177] Among them, represents the Gaussian smoothing operation, δ represents the standard deviation, and λ represents the image contrast factor.

[0178] When establishing the pyramid image, determine the total number of layers O of the pyramid and the number of blurred images S in each layer. For the s-th image in the o-th layer of the pyramid, its corresponding blurred scale parameter is

[0179] σ i σ(o,s) = σ02 o+s / S , o ∈ [0, 1, …, O - 1], s ∈ [0, 1, …, S - 1], i ∈ [0, 1, …, M - 1](10)

[0180] Among them, σ0 is the reference value of the scale space parameter, and M = O × S is the total number of images included in all scale spaces.

[0181] The mapping formula for replacing the scale parameter with the time unit is expressed as

[0182]

[0183] S32: Use the non-maximum suppression method to calculate the Hessian matrix value of each pixel point in the image pyramid, compare it with 26 pixel points in the same layer and the upper and lower adjacent layers; find the maximum value of the Hessian matrix after scale normalization and select it as the feature point.

[0184] Taking the feature point as the center on the gradient image, with a statistical range of radius 6σ, take the first-order differentials L x and L y of the feature point neighborhood for Gaussian weighting operation. Then rotate it around the origin with a 60° fan-shaped area and calculate the vector sum within this area, and take the longest direction as the main direction of the feature point. Use the local differential descriptor (M-LDB) to divide the feature point neighborhood into several grids, resample at intervals of scale σ in the sub-grids to obtain discrete points, calculate the average values of the brightness values, horizontal and vertical direction derivatives of the discrete points, and make the results of the binary test form a descriptor with a length of 3 bits.

[0185] Step S4: Taking the Hamming distance as the benchmark, match the local feature AKAZE descriptors of the image to be measured and the template image; perform data fitting on the successfully matched feature point pairs to solve the position change of the marker in the image to be measured; according to the calibration coefficient between the image pixels and the actual displacement, convert the position change in the marker image into the actual vibration displacement of the rotating component. Specifically, it includes:

[0186] S41: Use the Hamming distance to match the local feature AKAZE descriptors with the same length, and find the pixel coordinates of the same spatial point in the image to be measured and the template image respectively.

[0187] Through the local feature AKAZE matching, feature matching point pairs can be obtained, where (x1, y1), (x2, y2), …, (x q , y q )(q≥3) are the coordinates of the feature points of the reference image f(x i , y j ), and (x′1, y′1), (x′2, y′2), …, (x′ q , y′ q ) are the coordinates of the feature points matching them in the image to be measured g(x′ i , y′ j ).

[0188] S42: The geometric transformation between the rotor images is a full affine transformation, which can be expressed as follows:

[0189]

[0190] where t x and t y are the displacement amounts of the rotating component on the x-axis and y-axis respectively, which can characterize the vibration of the rotating component. Perform data fitting on the feature matching point pairs to solve t x and t y . According to the calibration coefficient K between the pixel position and the actual displacement, convert t x and t y into the vibration displacement in the actual space.

[0191]

[0192] As Figure 2 shown, an embodiment of the vibration monitoring device based on a mobile terminal and image processing of the present invention is applied:

[0193] Arrange a mobile terminal 2 equipped with a camera such as a drone, a smart phone or a Raspberry Pi on one side of the rotating component 1 to be measured.

[0194] The mobile terminal is equipped with a sequentially connected camera, an image acquisition module 3, a data storage module 4, and an image and signal processing module 5.

[0195] The image acquisition module 3 of the mobile terminal sets parameters such as the frame rate, resolution, exposure time, and shooting angle of the camera according to the actual working conditions, controls the camera to image the surface of the rotating part to be measured, and transmits the acquired image sequence to the data storage module 4; the data storage module 4 transmits the acquired image sequence to the image and signal processing module 5, performs operations such as similarity calculation, similarity signal spectrum refinement, local feature AKAZE detection and matching on the image sequence, and outputs the vibration displacement of the rotating part.

[0196] The camera of the mobile terminal takes pictures at a fixed frame rate, and the setting of the frame rate depends on the range of the vibration frequency of the rotating part. Generally, a relatively high shooting frame rate should be set to obtain higher measurement accuracy. The frame rate should be no less than 30 frames per second to prevent measurement failure caused by sampling aliasing.

[0197] The image area occupied by the rotating part is larger than 204,800 (640×480) pixels to provide sufficient texture information for similarity calculation of the image sequence and local feature detection of the image.

[0198] The exposure time of the camera of the mobile terminal should be controlled below 1000 μs to avoid excessive motion blur, so as to ensure that sufficient image features can be extracted; the illumination intensity of the shooting environment is above 300 lux to ensure good image quality and prevent low contrast caused by adverse lighting conditions. Generally, no additional lighting conditions are required.

[0199] The shooting angle of the camera of the mobile terminal is optional, generally preferably from -10 degrees to 10 degrees to ensure small image distortion and distortion; the shooting distance of the camera is set according to the actual size of the rotating part and the camera resolution, generally not less than 20 cm to ensure normal image focusing.

[0200] An apparatus embodiment applying the method of the present invention:

[0201] A computer device, which includes:

[0202] One or more processors;

[0203] A storage device for storing one or more programs;

[0204] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned vibration monitoring method based on a mobile terminal and image processing.

[0205] A computer medium embodiment applying the method of the present invention:

[0206] A computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, it implements the above-mentioned vibration monitoring method based on a mobile terminal and image processing.

[0207] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0208] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in one Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0209] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the specified functions in one Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0210] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in one Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0211] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent substitutions can still be made to the specific implementation manners of the present invention, and any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A vibration monitoring method based on a mobile terminal and image processing It is characterized in that it includes the following steps: Step 1: Obtain a number of images of the rotating component captured by the mobile terminal; The images are used to record the pose states of the rotating component at different times; Step 2: Screen the number of images in Step 1; The screening process is as follows: Select one image from the number of images as the template image, Select one or more images other than the template image as the images to be measured; Step 3: Construct an image similarity and spectrum correction model; The image similarity and spectrum correction model uses the normalized cross-correlation coefficient and the energy centroid correction method to estimate the rotation angle between the images to be measured and the template image in Step 2; Step 4: Rotate and reset the images to be measured according to the rotation angle calculated in Step 3 to form reset images; Step 5: Construct a non-linear diffusion filtering model to filter the reset images in Step 4 and the template images in Step 2 to obtain a non-linear scale space; After that, extract the feature points of the reset images and the template images in the non-linear scale space, and generate feature point descriptors for the extracted feature points according to their scale and direction information; Step 6: According to the feature descriptors in Step 5, use the Hamming distance as the benchmark to perform corresponding matching on the feature points in the images to be measured and the template images, and perform data fitting on the successfully matched feature point pairs to obtain the position change of the images to be measured relative to the template images; Step 7: According to the calibration coefficient between the image pixels and the actual displacement, convert the position change in Step 6 into the actual vibration displacement of the rotating component to realize the vibration monitoring of the rotating component; In Step 3, the construction method of the image similarity and spectrum correction model is as follows: S31: Calculate the normalized cross-correlation coefficient between each image in the images to be measured and the template image using the normalized cross-correlation algorithm, and convert the images to be measured into a one-dimensional signal S(k) representing image similarity; The calculation formula of the normalized cross-correlation coefficient is as follows: Among them, f(x i , y j ) is the template image, g(x i ′, y j ′) is the image to be measured, and m×n is the resolution; S32: Use the energy centroid correction method to perform spectrum correction on the one-dimensional signal S(k) representing image similarity in S31 to extract the high-precision rotation frequency f; The calculation formula of the corrected rotation frequency f is as follows: where G i is the amplitude of the i-th peak spectral line of the power spectrum of the one-dimensional signal S(k) characterizing the image similarity, k is the spectral line number corresponding to the point with the maximum amplitude, N is the length of the similarity signal, f s is the sampling frequency of the camera; S33: According to the relationship between the rotation frequency f in S32 and the acquisition time T of the images to be measured, obtain the rotation angle of the images to be measured relative to the template image. The calculation formula of the rotation angle is as follows: θ = f × T × 360°; In Step 5, the construction method of the non-linear diffusion filtering model is as follows: S51: Introduce a scale parameter, use the non-linear diffusion filtering equation to filter the reset images and the template images to construct a scale space; The calculation formula of the non-linear diffusion filtering equation is as follows: where (x, y) are the pixel coordinates of the image points, div represents divergence, is the gradient, L represents the image brightness, t is the scale parameter, c is the conduction diffusion function; The calculation formula of the conduction diffusion function is as follows: Among them, represents Gaussian smoothing operation, δ represents standard deviation, and λ represents image contrast factor; S52: According to the scale space in S51, establish a pyramid image model; The establishment process of the pyramid image model is as follows: Determine the total number of layers O of the pyramid and the number of blurred images S in each layer; For the s-th image in the o-th layer of the pyramid, determine the corresponding blur scale parameter; The calculation formula of the blur scale parameter is as follows: σ i (o,s) = σ02 o+s / S , o ∈ [0, 1, …, O - 1], s ∈ [0, 1, …, S - 1], i ∈ [0, 1, …, M - 1] (10) Where, σ0 is the reference value of the scale space parameter, and M = O × S is the total number of all images included in the scale space; Map the blur scale parameter to time units, and its mapping formula is as follows: S53: Use the non-maximum suppression method to calculate the Hessian matrix value of each pixel point in the pyramid image model in S52, and perform scale normalization processing to obtain the normalized Hessian matrix value; S54: Compare the normalized Hessian matrix value in S53 with 26 pixel points in the same layer and the upper and lower adjacent layers, find the maximum value of the Hessian matrix, and select it as the feature point; S55: Taking the feature point in S54 as the center, with a statistical range of radius 6σ, take the first-order differentials Lx and Ly of the feature point neighborhood for Gaussian weighting operation; and rotate it around the center with a 60° sector area, and calculate the vector sum within this area, and take the longest direction as the main direction of the feature point; S56: Use the local difference descriptor M-LDB to divide the neighborhood of the feature point in S55 into several sub-grid cells, and resample at intervals of scale σ in the sub-grid cells to obtain discrete points; S57: Calculate the brightness value of the discrete points in S56, and the average values of the derivatives in the horizontal and vertical directions, and let the average values of the two derivatives form a feature descriptor with a length of 3 bits; In the step 6, the matching method is as follows: Use the Hamming distance to match the feature descriptors with the same length, find the pixel coordinates of the same spatial point in the image to be measured and the template image respectively, obtain the feature matching point pairs, and perform geometric transformation between the template image and the image to be measured; The geometric transformation is a full affine transformation, and its calculation formula is as follows: Among them, (x1, y1), (x2, y2), …, (x q , y q )(q ≥ 3) are the pixel coordinates of the feature points of the reference image f(x i , y j ), and (x1′, y1′), (x2′, y2′), …, (x q ′, y q ′) are the pixel coordinates matching them in the image g(x i ′, y j ′) to be measured. t x and t y are the displacement amounts of the rotating component on the x-axis and y-axis respectively, characterizing the vibration of the rotating component. Perform data fitting on the feature matching point pairs to achieve the solution of t x and t y ; In step 7, the actual vibration displacement is obtained by multiplying the pixel position by the calibration coefficient K between the pixel position and the actual displacement to convert the time points t x and t y into the vibration displacement in the actual space. The calculation formula is as follows:

2. A vibration monitoring method based on a mobile terminal and image processing according to claim 1, characterized in that, In the step 1, the method for image acquisition is as follows: Use the camera of the mobile terminal to shoot the vibration video of the rotating component in different working states, and store it in the data storage module, and then process the vibration video to obtain several images; During the shooting process, the shooting frame rate of the camera remains unchanged, and the optical axis of the camera is perpendicular to the cross-section of the rotating component; The rotating component is an engine rotating shaft or a motor rotor or a steam turbine blade.

3. A vibration monitoring device based on a mobile terminal and image processing, characterized in that, Apply a vibration monitoring method based on a mobile terminal and image processing according to any one of claims 1-2; It has a mobile terminal device (2) that can be arranged on one side of the measured rotating component (1); The mobile terminal includes a camera, an image acquisition module (3), a data storage module (4), and an image and signal processing and module (5); The camera is used to shoot the moving surface of the rotating component; The image acquisition module (3) is used to set the frame rate, resolution, exposure time, and shooting angle of the camera according to the actual working conditions, control the camera to shoot the surface of the rotating component, and transmit the acquired image sequence to the data storage module (4); The data storage module (4) is used to store the image sequence; The image and signal processing module (5) is used to calculate the similarity of the image sequence, correct the energy center of gravity, detect and match the local feature AKAZE, and obtain the actual vibration displacement of the rotating component.

4. The vibration monitoring device based on a mobile terminal and image processing according to claim 3, wherein The camera takes pictures at a fixed frame rate, and the frame rate is set according to the vibration frequency range of the rotating component; The area occupied by the rotating component in the image is greater than or equal to 204,800 pixels.

5. The vibration monitoring device based on a mobile terminal and image processing according to claim 3, wherein The exposure time of the camera is less than 1000 μs; The light intensity is greater than 300 Lux; The shooting angle of the camera is from -10 degrees to 10 degrees; The shooting distance of the camera is greater than or equal to 20 cm.

6. The vibration monitoring device based on a mobile terminal and image processing according to claim 3, wherein The mobile terminal is a drone, a smart phone or a Raspberry Pi equipped with a camera.

Images