Multi-motor frequency measurement method and device based on GMA optical flow method and YOLOv8n, and storage medium

Abstract

The invention relates to a multi-motor frequency measurement method and device based on a GMA optical flow method and YOLOv8n and a storage medium. The method comprises the steps of obtaining a to-be-detected motor vibration video; acquiring a corresponding vibration intensity distribution image by using a GMA optical flow method based on deep learning; detecting by using the trained YOLOv8n model to obtain a motor and a corresponding frequency detection result; wherein the construction and training process of the YOLOv8n model comprises the following steps: constructing the YOLOv8n model, obtaining vibration intensity distribution images in different scenes, and constructing a motor target detection data set; configuring an experimental environment and initializing parameters of the YOLOv8n model; and training, verifying and testing the initialized YOLOv8n model by using a motor target detection data set. Compared with the prior art, the method has the advantages that the flexibility and the anti-interference performance of motor vibration frequency measurement are improved, and meanwhile, the measurement precision is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of motor frequency measurement, and in particular to a multi-motor frequency measurement method, device and storage medium based on GMA optical flow method and YOLOv8n. Background Art

[0002] A vibration motor is a DC motor with a bias mass connected to the shaft. As an excitation source, it is widely used in production and life, from small wearable devices to large industrial hoppers. In actual work, the failure of the vibration motor will affect the normal operation of the entire system, and effective fault detection is required. Motor vibration fault detection often uses a method based on vibration signals. The fault state of the vibration motor is analyzed by detecting the fault characteristic frequency. Therefore, the frequency measurement of the vibration motor is of great significance.

[0003] Motor vibration frequency measurement methods can be divided into two categories: contact and non-contact. The traditional contact method measures the motor's back electromotive force by designing a circuit, or installs Hall or photoelectric sensors to measure the motor's speed and frequency. This method requires the installation of additional equipment on the motor, lacks flexibility, and may affect the vibration characteristics of the equipment. The ultrasonic and laser rangefinder sensors required by the non-contact method are expensive and complex to set up. Based on this, the vision-based vibration measurement method is widely used in various vibration measurement studies because of its advantages such as non-contact, simple equipment requirements, and easy setup.

[0004] Vibration measurement methods based on vision can be divided into sparse point method and dense method. The sparse point method tracks and measures the motion of a small number of high-quality points by selecting feature points and other methods. However, the sparse point method needs to adjust the image processing steps and re-screen the points when the shooting angle and background lighting conditions change, and it relies on manual analysis of the spectrum, which has low flexibility. In the process of sparse point tracking, points may be lost or followed incorrectly, affecting the overall calculation accuracy. Because the dense method studies all pixels in the image, it can avoid the feature point selection step, and reduce the influence of a few erroneous points by calculating the global motion. However, the phase method commonly used in the dense method has poor estimation accuracy for slightly larger motions, and will be significantly disturbed when the overall picture shakes. It has high requirements for environmental settings, which limits its ease of use. Therefore, how to improve the flexibility and anti-interference of motor vibration frequency measurement while ensuring measurement accuracy has become a problem that needs to be solved in this field. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a multi-motor frequency measurement method, device and storage medium based on GMA optical flow method and YOLOv8n, which can improve the flexibility and anti-interference of motor vibration frequency measurement while ensuring measurement accuracy.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] According to a first aspect of the present invention, a multi-motor frequency measurement method based on a GMA optical flow method and YOLOv8n is provided, comprising the following steps: S1, obtaining a vibration video of a motor to be detected; S2, based on the vibration video of the motor to be detected, obtaining a corresponding vibration intensity distribution image using a deep learning-based GMA optical flow method; S3, based on the vibration intensity distribution image, using a trained YOLOv8n model to perform detection to obtain motor and corresponding frequency detection results; wherein the construction and training process of the YOLOv8n model comprises: S31, constructing a YOLOv8n model, obtaining vibration intensity distribution images under different scenarios, and constructing a motor target detection data set; S32, configuring an experimental environment and initializing parameters of the YOLOv8n model; S33, using the motor target detection data set to train, verify and test the initialized YOLOv8n model.

[0008] As a preferred technical solution, S2 specifically includes: extracting the optical flow field frame by frame from the motor vibration video to be detected, and arranging it in time sequence to obtain a vibration signal; calculating the vibration signal spectrum, and further calculating the global power spectrum; drawing a vibration intensity distribution image at each peak point frequency of the global power spectrum.

[0009] As a preferred technical solution, the optical flow field is extracted frame by frame from the motor vibration video to be detected, and the vibration signal is obtained by arranging it in time sequence, which specifically includes: taking the first frame as the reference frame, and sending each subsequent frame into a preset network, respectively calculating the displacement vector between the first frame and the first frame to obtain a dense optical flow field, wherein the dense optical flow field includes the displacement of all pixels in the first frame in each subsequent frame; and processing the displacement in the horizontal and vertical directions separately, and arranging the displacement of each frame in time sequence to obtain vibration signals in different directions.

[0010] As a preferred technical solution, the vibration signal spectrum is calculated, and the global power spectrum is further calculated, specifically including: performing FFT calculations on the vibration signals in different directions respectively to obtain their corresponding spectra; calculating the L2 norm of the spectra in each direction of all pixel points to obtain the global amplitude spectrum of the entire image, and further squaring it to obtain the power spectrum; adding the power spectra in different directions to obtain the global power spectrum, and the global power spectrum is used to express the overall picture movement.

[0011] As a preferred technical solution, a vibration intensity distribution image at each peak point frequency of the global power spectrum is drawn, specifically including: selecting a specific peak point frequency in the global power spectrum, extracting the spectrum amplitude of each pixel movement at the frequency, and drawing the spectrum amplitude into a vibration intensity distribution image according to the spatial position of the pixel. The shape of the motor will be displayed in the vibration intensity distribution image corresponding to the vibration frequency of the motor.

[0012] As a preferred technical solution, the vibration video of the motor to be detected is obtained by using a slow-motion shooting mode of a smart phone.

[0013] As a preferred technical solution, vibration intensity distribution images in different scenes are obtained to construct a motor target detection dataset, specifically including: shooting multiple motor vibration videos under different background and lighting conditions; based on the multiple motor vibration videos, using the GMA optical flow method based on deep learning to respectively obtain corresponding vibration intensity distribution images; marking the motor targets in the multiple vibration intensity distribution images to obtain a motor target detection dataset.

[0014] As a preferred technical solution, S32 specifically includes: using Pytorch as the network framework, with an initial learning rate of 0.01, a batch size of 16, and a number of iterations of 100.

[0015] According to a second aspect of the present invention, a multi-motor frequency measurement device based on a GMA optical flow method and YOLOv8n is provided, comprising a memory, a processor, and a program stored in the memory, wherein the method described is implemented when the processor executes the program.

[0016] According to a third aspect of the present invention, there is provided a storage medium having a program stored thereon, wherein the program implements the method described above when executed.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. The present invention uses the GMA optical flow method based on deep learning to obtain the corresponding vibration intensity distribution image, and combines the YOLOv8n model to detect the motor target from the vibration intensity distribution image to achieve multi-motor frequency measurement, which can improve the flexibility and automation of motor vibration frequency measurement, and the trained and tested YOLOv8n model can ensure detection accuracy;

[0019] 2. The present invention uses the GMA optical flow method based on deep learning to extract the dense optical flow field between frames from the motor vibration video to be detected, which can effectively enhance the anti-interference ability;

[0020] 3. The present invention uses easily available smart phones to shoot videos, which can reduce equipment costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of the implementation flow of the method provided in the embodiment of the present invention;

[0022] Figure 2 A schematic diagram of a frequency intensity distribution image provided in an embodiment of the present invention;

[0023] Figure 3 A log power spectrum diagram of a test video provided in an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of target detection results of a vibration intensity distribution image at each frequency provided in an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of a final result of frequency detection provided in an embodiment of the present invention;

[0026] in: Figure 4 Part (a) corresponds to a frequency of 6.0 Hz, part (b) corresponds to a frequency of 48.4 Hz, part (c) corresponds to a frequency of 61.6 Hz, part (d) corresponds to a frequency of 96.4 Hz, part (e) corresponds to a frequency of 122.8 Hz, part (f) corresponds to a frequency of 144.8 Hz, part (g) corresponds to a frequency of 184.4 Hz, part (h) corresponds to a frequency of 193.2 Hz, and part (i) corresponds to a frequency of 238.8 Hz. DETAILED DESCRIPTION

[0027] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0028] Example

[0029] The multi-motor frequency measurement method based on the GMA optical flow method and YOLOv8n provided by the present invention has simple equipment requirements, strong anti-interference and high degree of automation. Among them, GMA refers to the global motion aggregation module, and the optical flow method provided by the present invention is implemented based on the GMA module.

[0030] like Figure 1 As shown, this embodiment provides a multi-motor frequency measurement method based on GMA optical flow method and YOLOv8n, and the specific implementation steps are as follows:

[0031] Step S1, obtaining a video of the motor vibration to be detected. Optionally, a smartphone that is easily available is used to shoot the motor vibration video through its slow motion shooting mode, instead of a traditional high-speed camera.

[0032] Step S2: Based on the vibration video of the motor to be detected, a corresponding vibration intensity distribution image is obtained using a GMA optical flow method based on deep learning.

[0033] First, extract the optical flow field frame by frame from the motor vibration video to be detected, and arrange it in time sequence to obtain the vibration signal. Specifically, use the GMA optical flow method based on deep learning to extract the optical flow field frame by frame from the captured video:

[0034] GMA accepts two frames of RGB images I 1 and I 2 As input, and finally output the optical flow field H and W are the height and width of the input image, which correspond to the number of pixels in the vertical and horizontal directions. After receiving the input, the convolutional neural network with shared weights extracts features from the two frames of images and reduces the resolution to 1 / 8 scale, thus obtaining the feature map F. 1 , H' and W' are the height and width of the image after feature extraction, respectively. Then, the dot product is calculated between all feature vectors of the two feature maps as the similarity to obtain the correlation volume. Then the last two dimensions of the correlation body are reduced multiple times to obtain a multi-scale correlation body, and similarity information is extracted from it through a fixed-size search box. The correlation body calculation formula is as follows:

[0035]

[0036] In the formula, the subscript Respectively represent F 1 and F 2 The height and width dimensions, Represents the feature map channel dimension, and D is the number of channels in the feature map.

[0037] GMA further uses a global motion aggregation module based on self-attention in the context branch to aggregate motion features. The algorithm uses the context feature F extracted by convolution. c The self-similarity calculation is performed to obtain the attention matrix, which is then combined with the motion feature F m Combined, we get the motion features after aggregating the global motion information The formula is as follows:

[0038]

[0039] Where α is a learnable weight that is initially 0, allowing the model to choose between global and local features, q, k, v are 1x1 convolutions with different weights, used to map feature maps to queries, keys, and values ​​in attention calculations, respectively, and D is the number of channels in the feature map.

[0040] Subsequently, the context features are connected to the motion features before and after aggregation and input into the convolutional gated recurrent unit for cyclic update prediction. Each update output Δf is added to the optical flow obtained from the previous update (initialized to 0) as the input for the next update to guide the movement of the search box. After the cyclic update reaches the specified number of times, the final predicted optical flow is output.

[0041] When processing vibration video, the first frame I 1 As a reference frame, each subsequent frame I t Send it into the network and calculate it respectively with I 1 The displacement vector between them is used to obtain the dense optical flow field f t (x,y), the optical flow field contains I 1 All pixels in I t The displacement in the horizontal and vertical directions is processed separately, and the displacement of each frame is arranged in time sequence to obtain the vibration signal Δx t (x,y) and Δy t (x,y).

[0042] Secondly, the vibration signal spectrum is calculated, and the global power spectrum is further calculated.

[0043] The vibration signals Δx in two directions are t (x,y) and Δy t (x,y) performs FFT calculation to obtain the spectrum and Then, the L2 norm of the x and y direction spectra of all pixels is calculated to obtain the global amplitude spectrum of the entire image, and the power spectrum is further squared. Finally, the power spectra in the two directions are added together to express the global power spectrum of the overall picture motion. The calculation formula of the global power spectrum P is as follows:

[0044]

[0045] Finally, the vibration intensity distribution image at each peak frequency of the global power spectrum is plotted.

[0046] Select a specific peak frequency l in the global power spectrum P * , extract the frequency spectrum amplitude of each pixel motion at this frequency These amplitudes are plotted into a vibration intensity distribution image according to the spatial position of the pixels, where the darker the color, the higher the vibration intensity. When the vibration frequency of the motor is selected, the shape of the motor will be displayed in the corresponding vibration intensity distribution image (such as Figure 2 shown).

[0047] Step S3, based on the vibration intensity distribution image, use the trained YOLOv8n model to perform detection to obtain the motor and corresponding frequency detection results.

[0048] Among them, the construction and training process of the YOLOv8n model includes:

[0049] Step S31, constructing a YOLOv8n model, obtaining vibration intensity distribution images under different scenarios, and constructing a motor target detection data set.

[0050] Specifically, under different background and lighting conditions, multiple motor vibration videos were shot to extract optical flow and calculate global power spectrum, and vibration intensity distribution images at each peak frequency were collected. The motor targets were marked, and a total of 249 images were obtained, including 32 positive samples. The total number was expanded to 531 through flipping operations to obtain a motor target detection dataset, which was divided into training set, validation set, and test set in a ratio of 8:1:1.

[0051] Step S32, configure the experimental environment and initialize the parameters of the YOLOv8n model.

[0052] The network framework used in the experiment is Pytorch, the initial learning rate of the YOLOv8n model is 0.01, the batch size is 16, and the number of iterations is 100.

[0053] Step S33, using the motor target detection dataset to train, verify and test the initialized YOLOv8n model.

[0054] Specifically, the motor target detection data set obtained in step S31 is used to train the YOLOv8n model initialized in step S32 to detect the motor target. After each training, the validation set is used to verify the effect of the model to prevent overfitting problems. After the training is completed, the target detection model is tested using the test set in the data set. The average mean precision mAP (mean Average Precision) detected by the test model is 0.796, which can meet the detection requirements.

[0055] In actual application, steps S1 to S3 are executed, and the YOLOv8n model trained in steps S31 to S33 is used to detect the motors in the input vibration intensity distribution images. When the motor is detected, the corresponding frequency and the detection frame are displayed to the first frame. Specifically:

[0056] Shoot the vibration video of the motor to be detected and calculate the global power spectrum. The global power spectrum is as follows: Figure 3 As shown in the figure, the peak frequency is marked. The vibration intensity distribution images at each peak frequency in the global power spectrum are plotted respectively, and the motor detection is performed on these images using the trained YOLOv8n model. The results are shown in the figure. Figure 4 As shown, 9 images are respectively Figure 3The 9 frequencies in correspondence, the trained YOLOv8n is only Figure 4 The motor is detected in part (b) (48.4Hz) and part (c) (61.6Hz). When the motor is detected, the frequency corresponding to the image and the detection frame are displayed in the first frame of the video to realize the frequency detection of the motor. The final result is as follows Figure 5 As shown, the detection boxes and frequencies in the figure come from Figure 4 Part (b) and (c) of the

[0057] Among them, the experimental environment needs to be configured for actual application, and the experimental parameters are set according to the resolution, sampling rate and sampling rate of the smartphone camera. Optionally, the shooting mode of the smartphone is set to 720P@480FPS, and the output native video resolution is 1280x720. After shooting, the resolution is reduced to 640x360 to reduce the memory usage during calculation. The sampling rate of the video shot by the mobile phone is 480Hz, the sampling rate of the accelerometer is 200Hz, and the number of video frames analyzed by the optical flow method is set to 1200 frames. In order to reduce the impact when the video starts shooting, it is intercepted from the 500th frame. In terms of the accelerometer, only the x and y axis data are collected to correspond to the optical flow method, and in order to align the spectral resolution of the two, the number of analysis points of the accelerometer is set to 500 points.

[0058] To verify the effectiveness of the proposed method, multiple motor vibration videos can be shot to measure the frequency, compared with the measurement results of the accelerometer, the frequency measurement error rate of the test method can be tested, and finally a video can be shot to verify the overall process. Optionally, three sets of motor vibration videos are shot, and the motor in each video vibrates at a different frequency. The frequency is measured using the method proposed in this embodiment and compared with the measurement results of the accelerometer. After testing, the average frequency measurement error rate of the method is 0.97%.

[0059] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A multi-motor frequency measurement method based on GMA optical flow method and YOLOv8n, characterized in that: The following steps are involved: S1, obtaining the vibration video of the motor to be detected; S2, based on the vibration video of the motor to be detected, using the GMA optical flow method based on deep learning to obtain a corresponding vibration intensity distribution image; S3, based on the vibration intensity distribution image, using the trained YOLOv8n model to perform detection to obtain the motor and corresponding frequency detection results; The construction and training process of the YOLOv8n model includes: S31, build the YOLOv8n model, obtain the vibration intensity distribution images under different scenes, and build the motor target detection dataset; S32, configure the experimental environment and initialize the parameters of the YOLOv8n model; S33, using the motor target detection dataset to train, verify and test the initialized YOLOv8n model.

2. The multi-motor frequency measurement method based on GMA optical flow method and YOLOv8n according to claim 1, characterized in that, The S2 specifically includes: Extracting the optical flow field frame by frame from the motor vibration video to be detected, and arranging it in time sequence to obtain a vibration signal; Calculate the vibration signal spectrum and further calculate the global power spectrum; Draw the vibration intensity distribution image at each peak frequency of the global power spectrum.

3. The multi-motor frequency measurement method based on GMA optical flow method and YOLOv8n according to claim 2, characterized in that, Extracting the optical flow field frame by frame from the motor vibration video to be detected and arranging it in time sequence to obtain the vibration signal, specifically including: The first frame is used as a reference frame, and each subsequent frame is sent to a preset network to calculate the displacement vector between the first frame and the frame to obtain a dense optical flow field, wherein the dense optical flow field includes the displacement of all pixels in the first frame in each subsequent frame; The displacements in the horizontal and vertical directions are processed separately, and the displacements of each frame are arranged in time sequence to obtain vibration signals in different directions.

4. The multi-motor frequency measurement method based on GMA optical flow method and YOLOv8n according to claim 3, characterized in that, Calculate the vibration signal spectrum and further calculate the global power spectrum, including: Perform FFT calculations on vibration signals in different directions to obtain their corresponding spectra; Calculate the L2 norm of the frequency spectrum of all pixels in each direction to obtain the global amplitude spectrum of the entire image, and further square it to obtain the power spectrum; The power spectra in different directions are added together to obtain a global power spectrum, which is used to express the overall picture motion.

5. The multi-motor frequency measurement method based on GMA optical flow method and YOLOv8n according to claim 3, characterized in that, Draw the vibration intensity distribution image at each peak frequency of the global power spectrum, including: A specific peak point frequency in the global power spectrum is selected, and the spectrum amplitude of each pixel movement at this frequency is extracted. The spectrum amplitude is plotted into a vibration intensity distribution image according to the spatial position of the pixel. The shape of the motor will be displayed in the vibration intensity distribution image corresponding to the motor vibration frequency.

6. The multi-motor frequency measurement method based on GMA optical flow method and YOLOv8n according to claim 1, characterized in that, The motor vibration video to be detected is obtained by using a slow motion shooting mode of a smart phone.

7. The multi-motor frequency measurement method based on GMA optical flow method and YOLOv8n according to claim 1, characterized in that: Obtain vibration intensity distribution images in different scenarios and build a motor target detection dataset, including: Take multiple videos of motor vibrations under different backgrounds and lighting conditions; Based on the multiple motor vibration videos, corresponding vibration intensity distribution images are respectively acquired using a GMA optical flow method based on deep learning; The motor targets in multiple vibration intensity distribution images are marked to obtain a motor target detection dataset.

8. The multi-motor frequency measurement method based on GMA optical flow method and YOLOv8n according to claim 1, characterized in that: The S32 specifically includes: using Pytorch as the network framework, with an initial learning rate of 0.01, a batch size of 16, and a number of iterations of 100.

9. A multi-motor frequency measurement device based on GMA optical flow method and YOLOv8n, comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A storage medium having a program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 8 is implemented.

Images