Motor vibration detection method and device, storage medium and computer equipment

Abstract

The invention provides a motor vibration detection method and device, a storage medium and computer equipment, and the method comprises the steps: segmenting a motor object in each frame of image in a to-be-detected video through employing a video segmentation model, and obtaining a motor image set formed by motor images corresponding to each frame of image, therefore, interference except the motor object in the image can be eliminated, and the precision of motor vibration detection is improved. Then, after feature point extraction is carried out on each motor image, a point pair set is determined according to the multiple feature points of the adjacent motor images, then the similarity of each pair of points in the point pair set is calculated, N pair points with the highest similarity are selected for probability distribution modeling, and therefore the pair points with high quality and high matching degree can be screened out; therefore, the accuracy of the obtained discrete Gaussian distribution sequence is improved, finally, vibration characteristic extraction and mapping can be performed on the Gaussian distribution sequence to obtain the vibration frequency and the vibration amplitude, and the precision of motor vibration detection is further improved.

Description

Technical Field

[0001] The present application relates to the field of vibration detection technology, and in particular to a motor vibration detection method, device, storage medium and computer equipment. Background Art

[0002] Testing the running frequency of the motor can effectively judge the running status of the point, so as to give early warning of the abnormal state of the motor in time. GB10068-2000 "Vibration Limits and Test Methods for Rotating Motors" stipulates the vibration limits, measurement methods and judgment standards of rigid foundations for motors with different center heights on rigid foundations. According to this standard, it can be judged whether the motor meets the standard. Therefore, measuring the vibration frequency of the motor has important application value in the field of motor abnormality diagnosis.

[0003] The traditional motor frequency measurement scheme requires the installation of vibration measurement sensors on the motor, such as displacement sensors, acceleration sensors, etc., in order to calculate the vibration frequency of the motor. However, there are many inconveniences in installing sensors on the motor, especially some contact sensors may interfere with the normal operation of the motor. In order to overcome the defects brought by contact vibration detection, the vibration of the motor can be detected by non-contact detection methods. However, when non-contact vibration detection is applied, the accuracy of the vibration detection of the motor is low due to the large amount of invalid information in the collected data. Summary of the invention

[0004] The purpose of the present application is to solve at least one of the above-mentioned technical defects, especially the technical defect in the prior art that when non-contact vibration detection is applied, the accuracy of vibration detection of the motor is low due to the large amount of invalid information in the collected data.

[0005] In a first aspect, the present application provides a motor vibration detection method, the method comprising:

[0006] Get the video to be detected;

[0007] Segmenting the motor object in each frame image in the video to be detected by using a preset video segmentation model to obtain a motor image set formed by the motor images corresponding to each frame image;

[0008] Extracting feature points from each motor image in the motor image set to obtain a plurality of feature points corresponding to each motor image, and determining a point pair set corresponding to the motor image set according to a plurality of feature points of adjacent motor images in the motor image set;

[0009] Calculate the similarity of each pair of points in the point pair set respectively, and select the N pairs of points with the highest similarity to form a feature point pair set;

[0010] Probability distribution modeling is performed on the feature point pair set to obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set, and then vibration features are extracted from the discrete Gaussian distribution sequence and mapped to obtain vibration frequency and vibration amplitude.

[0011] In one embodiment, the method of using a preset video segmentation model to segment the motor object in each frame of the video to be detected to obtain a motor image set formed by the motor images corresponding to each frame of the video includes:

[0012] Determine a preset video segmentation model;

[0013] Dividing the video to be detected into picture frames according to the acquisition frame rate of the video to be detected, so as to obtain each frame image in the video to be detected;

[0014] Acquire segmentation cue points, and input the segmentation cue points and each frame image into the video segmentation model, so as to segment the motor object in each frame image according to the segmentation cue points, and obtain the motor image corresponding to each frame image;

[0015] A motor image set is generated based on the individual motor images.

[0016] In one embodiment, extracting vibration features from the Gaussian distribution sequence and mapping to obtain vibration frequency and vibration amplitude includes:

[0017] Determine a preset vibration detection model; the vibration detection model includes a multi-layer convolutional network and a multi-layer perceptron module;

[0018] Using the multi-layer convolutional network to extract features from the Gaussian distribution sequence to obtain a feature matrix, and straightening the feature matrix to obtain vibration features;

[0019] The vibration characteristics are input into the sensor module for feature mapping to obtain the vibration frequency and vibration amplitude.

[0020] In one embodiment, extracting feature points from each motor image in the motor image set to obtain a plurality of feature points corresponding to each motor image includes:

[0021] Determine a threshold parameter for each area in the motor image according to an image parameter of each motor image in the motor image set;

[0022] Preprocessing each motor image to highlight the main features of each motor image;

[0023] For each preprocessed motor image, traverse each pixel of the motor image, if the absolute value of the difference between the grayscale value of the surrounding pixels of the pixel and the pixel is less than the threshold parameter of the area where the pixel is located, and the surrounding pixels are greater than the preset threshold, then the pixel is determined as a feature point, and after traversing the motor image, multiple feature points corresponding to the motor image are obtained;

[0024] After traversing each motor image, multiple feature points corresponding to each motor image are obtained.

[0025] In one embodiment, determining a point pair set corresponding to the motor image set according to a plurality of feature points of adjacent motor images in the motor image set includes:

[0026] For each motor image except the last motor image in the motor image set, each feature point corresponding to the motor image is matched with each feature point in the next motor image of the motor image, so as to determine the feature point corresponding to each feature point corresponding to the motor image in the next motor image, and then form a point pair subset of the motor image and the next motor image; the point pair subset includes a plurality of pairs of points, each pair of points includes two feature points;

[0027] After determining the point pair subsets of each pair of adjacent motor images in the motor image set, a point pair set of the motor image set is generated according to the point pair subsets of each pair of adjacent motor images in the motor image set.

[0028] In one embodiment, respectively calculating the similarity of each pair of points in the point pair set includes:

[0029] For each pair of points in the point pair set, determining a descriptor of a feature point in the pair of points;

[0030] The Hamming distance between the feature points in the pair of points is calculated according to the descriptors of the feature points in the pair of points, and the similarity between the feature points in the pair of points is determined according to the Hamming distance between the feature points in the pair of points.

[0031] In one embodiment, performing probability distribution modeling on the feature point pair set to obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set includes:

[0032] Calculate the offset value of each feature pair point in the feature point pair set;

[0033] According to the offset value of each feature point pair in the feature point pair set, an offset sequence corresponding to the feature point pair set is formed;

[0034] The offset sequence is quantized, and discrete Gaussian distribution modeling is performed on the quantized offset sequence based on a preset Gaussian function to obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set.

[0035] In a second aspect, the present application provides a motor vibration detection device, the device comprising:

[0036] A video acquisition module is used to acquire the video to be detected;

[0037] An image segmentation module is used to segment the motor object in each frame image in the video to be detected by using a preset video segmentation model to obtain a motor image set formed by the motor images corresponding to each frame image;

[0038] A feature point extraction module, used to extract feature points from each motor image in the motor image set to obtain a plurality of feature points corresponding to each motor image, and determine a point pair set corresponding to the motor image set according to a plurality of feature points of adjacent motor images in the motor image set;

[0039] A similarity calculation module is used to calculate the similarity of each pair of points in the point pair set, and select the N pairs of points with the highest similarity to form a feature point pair set;

[0040] The vibration detection module is used to perform probability distribution modeling on the feature point pair set, obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set, extract vibration features from the discrete Gaussian distribution sequence and map it to obtain vibration frequency and vibration amplitude.

[0041] In a third aspect, the present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the motor vibration detection method described in any of the above embodiments.

[0042] In a fourth aspect, the present application provides a computer device, comprising: one or more processors, and a memory;

[0043] The memory stores computer-readable instructions, and when the one or more processors execute the computer-readable instructions, the steps of the motor vibration detection method described in any one of the above embodiments are performed.

[0044] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0045] The motor vibration detection method, device, storage medium and computer equipment provided by the present application include: using a preset video segmentation model to segment the motor object in each frame image of the video to be detected, and after segmentation, a motor image set formed by the motor image corresponding to each frame image can be obtained, so that the interference of other objects or backgrounds in the image other than the motor object on the vibration detection can be eliminated, thereby improving the accuracy of the motor vibration detection. Then, after extracting the feature points of each motor image, a point pair set is determined according to multiple feature points of adjacent motor images, and then the similarity of each pair of points in the point pair set is calculated respectively, and the N pairs of points with the highest similarity are selected for probability distribution modeling, so that high-quality and high-matching pairs of points can be screened out, thereby improving the accuracy of the obtained discrete Gaussian distribution sequence, and finally the Gaussian distribution sequence can be subjected to vibration feature extraction and mapping to obtain high-accuracy vibration frequency and vibration amplitude, so as to achieve the purpose of further improving the accuracy of motor vibration detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 A schematic diagram of a flow chart of a motor vibration detection method provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of a process for segmenting a motor object in each frame of an image to be detected in a video using a preset video segmentation model provided in an embodiment of the present application;

[0049] Figure 3 A schematic diagram of a process for extracting vibration features from a Gaussian distribution sequence and mapping the vibration frequency and vibration amplitude provided in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of the structure of a motor vibration detection device provided in an embodiment of the present application;

[0051] Figure 5 An internal structure diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] In one embodiment, the present application provides a motor vibration detection method, and the following embodiment is described by applying the method to a server. It can be understood that the motor vibration detection method can be executed by a single server or a server cluster composed of multiple servers, and the present application does not make specific restrictions on this.

[0054] like Figure 1 As shown, the present application provides a motor vibration detection method, the method comprising:

[0055] S101: Obtain a video to be detected.

[0056] The video to be tested refers to a video obtained by shooting the motor in a working state.

[0057] In this step, when the user needs to detect whether the motor operating status is abnormal, a motor vibration detection command can be initiated through the client. When the server receives the motor vibration detection command, the server will obtain a video of the motor in the working state based on the motor vibration detection command, that is, the video to be detected.

[0058] Specifically, the user can upload the filmed video to be detected to the client, and then initiate a motor vibration detection instruction. When the server receives the motor vibration detection instruction, it determines and obtains the video to be detected uploaded by the user according to the motor vibration detection instruction. The user can also directly initiate a motor vibration detection instruction. When the server receives the motor vibration detection instruction, it controls the pre-set shooting device to record the motor to obtain the video to be detected. The user can also specify a video to be detected in a certain time period that has been filmed and initiate a motor vibration detection instruction. When the server receives the motor vibration detection instruction, it obtains the time period specified by the user and obtains the video to be detected in the time period from the cloud or local database. The specific method of obtaining the video to be detected can be determined according to the actual situation, and this application does not impose specific restrictions on this.

[0059] Furthermore, after obtaining the video to be detected, it can be checked whether the number of frames contained in the video to be detected meets the detection requirements. If the number of frames contained in the video to be detected does not meet the detection requirements, a prompt can be triggered. If the number of frames contained in the video to be detected meets the detection requirements, a video composed of L frames of images can be segmented from the video to be detected, and the video can be used as the final video to be detected. Wherein, L is an empirical value, and its setting can take into account factors such as the effect of motor vibration detection, the performance of computer equipment, etc. It can be understood that when the number of frames contained in the video to be detected does not meet the detection requirements, it means that the number of frames contained in the video to be detected is less, which may cause the accuracy of vibration detection of the motor to be decreased, so the system prompt can be triggered at this time to re-acquire the video to be detected.

[0060] S102: Segment the motor object in each frame image in the video to be detected by using a preset video segmentation model to obtain a motor image set formed by the motor images corresponding to each frame image.

[0061] Among them, the video segmentation model is used to identify and segment the motor object in each frame image in the video to be detected, obtain the motor image corresponding to each frame image, and output a motor image set formed by the motor images corresponding to each frame image.

[0062] In this step, the video segmentation model can be used to analyze each frame image in the video to be detected, and then the motor object in each frame image can be identified, and the motor object in each frame image can be segmented to obtain the motor image corresponding to each frame image, and a motor image set can be generated according to the motor images corresponding to each frame image.

[0063] In one example, the video segmentation model may adopt a zero-sample video segmentation model. When the zero-sample video segmentation model is used as the video segmentation model in the present application, since the zero-sample video segmentation model does not require specific training, it can save computing resources and is easy to deploy, thereby improving the efficiency of motor vibration detection.

[0064] S103: extracting feature points from each motor image in the motor image set to obtain a plurality of feature points corresponding to each motor image, and determining a point pair set corresponding to the motor image set according to a plurality of feature points of adjacent motor images in the motor image set.

[0065] Among them, feature points refer to pixel points with obvious features in the motor image, which can be used to describe the local features of the image. Feature points include but are not limited to corner points, edge points, etc.

[0066] In this step, when determining the motor image set, image processing and feature extraction can be performed on each motor image in the motor image set to determine multiple feature points corresponding to each motor image, thereby achieving the purpose of extracting feature points from the motor image. Then, a set of corresponding point pairs of the motor image set can be determined based on multiple feature points corresponding to motor images with adjacent frame numbers in the motor image set.

[0067] Specifically, the point pair set includes multiple pairs of points, each pair of points consists of two feature points, and the feature points of each pair of points are derived from feature points in adjacent motor images. In an example, a subset of the point pair set can be expressed as: {[( ), ( )], [( ), ( )] …, [( ), ( ) ]},in, Indicates the frame number. Indicates For feature points, [( ), ( )] represents a pair of points, i.e. Frame motor image and The first For feature points. It should be noted that the above Frame motor image and Taking the frame motor image as an example, in fact, the point pair set includes multiple pairs of feature points formed by each pair of adjacent motor images in the motor image set.

[0068] It can be understood that since the motor images in the motor image set are all images of the same motor taken at continuous time, and then the feature points of each motor image are extracted in the same way, there is a corresponding relationship between the feature points in each motor image. For example, assuming that there are adjacent motor images A, motor image B and motor image C, any feature point in motor image A can find a unique corresponding matching feature point in motor image B. Similarly, any feature point in motor image B can also find a unique corresponding matching feature point in motor image C.

[0069] S104: Calculate the similarity of each pair of points in the point pair set respectively, and select the N pairs of points with the highest similarity to form a feature point pair set.

[0070] The similarity is used to measure the quality of point matching, that is, to measure the matching degree between two feature points in the point pair. N is an empirical value and can be adjusted according to actual needs and data volume. This application does not impose specific restrictions on this.

[0071] In this step, the matching degree between the two feature points in each pair of points is measured by calculating the similarity of each pair of points in the point pair set, and the N pairs of points with the highest similarity are selected, and a feature point pair set is generated based on the selected pairs of points. This can further improve the quality of the point pair set, so that when the data is subsequently used to calculate the vibration frequency and vibration amplitude, the accuracy of the vibration detection can be improved due to the high accuracy of the data used.

[0072] S105: Probability distribution modeling is performed on the feature point pair set to obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set, and then vibration features are extracted from the discrete Gaussian distribution sequence and mapped to obtain vibration frequency and vibration amplitude.

[0073] In this step, when the feature point pair set is determined, its offset sequence is determined according to the feature point pair set, and then the probability distribution modeling is performed on the offset sequence to obtain the discrete Gaussian distribution sequence corresponding to the feature point pair set, and finally, the vibration frequency and vibration amplitude of the motor corresponding to the video to be detected are obtained by extracting vibration features from the discrete Gaussian distribution sequence and mapping the extracted vibration features to the frequency and amplitude space. Among them, the offset sequence refers to a sequence composed of the offset values ​​of each pair of points in the feature point pair set.

[0074] It can be understood that the vibration frequency and vibration amplitude are determined by extracting the vibration features and then mapping the vibration features to the frequency and amplitude controls. In this way, effective information can be extracted as much as possible, and noise interference in the calculation of the final vibration frequency and vibration amplitude can be avoided, so as to improve the robustness and detection accuracy of vibration detection.

[0075] The motor vibration detection method, device, storage medium and computer equipment provided by the present application include: using a preset video segmentation model to segment the motor object in each frame image of the video to be detected, and after segmentation, a motor image set formed by the motor image corresponding to each frame image can be obtained, so that the interference of other objects or backgrounds in the image other than the motor object on the vibration detection can be eliminated, thereby improving the accuracy of the motor vibration detection. Then, after extracting the feature points of each motor image, a point pair set is determined according to multiple feature points of adjacent motor images, and then the similarity of each pair of points in the point pair set is calculated respectively, and the N pairs of points with the highest similarity are selected for probability distribution modeling, so that high-quality and high-matching pairs of points can be screened out, thereby improving the accuracy of the obtained discrete Gaussian distribution sequence, and finally the Gaussian distribution sequence can be subjected to vibration feature extraction and mapping to obtain high-accuracy vibration frequency and vibration amplitude, so as to achieve the purpose of further improving the accuracy of motor vibration detection.

[0076] like Figure 2As shown, in one embodiment, a motor object in each frame image in the video to be detected is segmented using a preset video segmentation model to obtain a motor image set formed by motor images corresponding to each frame image, including:

[0077] S201: Determine a preset video segmentation model.

[0078] In one example, the video segmentation model in this embodiment can adopt the SAM2 (Segment Anything in Images and Videos, zero-sample video segmentation) model. The SAM2 model is a high-precision zero-sample video segmentation model. When using the SAM2 model as the video segmentation model in this embodiment, since the recognition and segmentation tasks can be achieved without the need to perform segmentation task learning for specific motor samples, computing resources can be saved, and the efficiency and performance of motor vibration detection can also be improved.

[0079] S202: Segment the video to be detected into picture frames according to the acquisition frame rate of the video to be detected, so as to obtain each frame image in the video to be detected.

[0080] In this step, the sampling frame rate of the shooting device is obtained, and then the video to be detected is segmented into picture frames according to the sampling frame rate to obtain each frame image in the video to be detected.

[0081] S203: Acquire segmentation cue points, and input the segmentation cue points and each frame image into a video segmentation model, so as to segment the motor object in each frame image according to the segmentation cue points, and obtain a motor image corresponding to each frame image.

[0082] The segmentation cue points refer to cues used to prompt the model to identify and segment motor objects.

[0083] In this step, the segmentation cue points are used as model cues and each frame image in the video to be detected is used as model input. The segmentation cue points and each frame image are input into the video segmentation model. In this way, the video segmentation model can identify and segment the motor objects in each frame image according to the segmentation cue points, and obtain a motor image corresponding to each frame image that only retains the motor object.

[0084] In an example, when the SAM2 model is used as a video segmentation model, the SAM2 model can be loaded and a predictor build_sam2_video_predictor can be built, and then the segmentation cue points are obtained, the segmentation cue points and each frame image are input into the SAM2 model for inference, and the segmentation cue points are propagated in the SAM2 model to obtain the mask, and finally the motor segmentation mask map of each frame image is obtained, and then the motor image of each frame image is determined.

[0085] S204: Generate a motor image set based on each motor image.

[0086] It can be understood that the video segmentation model is used to identify and segment the motor object in each frame image in the video to be detected. After segmentation, a motor image set formed by the motor images corresponding to each frame image can be obtained. In this way, interference with vibration detection caused by other objects or backgrounds in the image except the motor object can be eliminated, thereby improving the accuracy of motor vibration detection.

[0087] like Figure 3 As shown, in one embodiment, the vibration feature extraction of the Gaussian distribution sequence is performed and mapped to obtain the vibration frequency and vibration amplitude, including:

[0088] S301: Determine a preset vibration detection model.

[0089] The vibration detection model includes a multi-layer convolutional network and a multi-layer perceptron module. Specifically, the multi-layer convolutional network is a deep learning model, and the multi-layer perceptron module includes two multi-layer perceptrons. The multi-layer perceptron is a feedforward artificial neural network model. The multi-layer perceptron consists of multiple layers, including an input layer, a hidden layer, and an output layer. Each layer consists of multiple neurons, and the neurons are connected by weights.

[0090] Specifically, the vibration detection model of the present application can be obtained by using the discrete Gaussian distribution sequence obtained in the process of motor vibration detection as training samples, the vibration frequency and vibration amplitude of the motor as sample labels, and using the loss function to train the preset pre-trained model. During the training process, the server can input the training samples and their corresponding sample labels into the pre-trained model for forward propagation, and use the loss function in the back propagation of the pre-trained model to adjust the parameters of the pre-trained model until the pre-trained model meets certain iteration conditions. The pre-trained model that has been trained can be used as a vibration detection model. It can be understood that the iteration condition can be reaching a certain number of iterations, or it can be that the parameter change is less than a preset threshold value, and the present application does not make specific restrictions on this. In addition, in the process of training the vibration detection model, the sample labels can be collected by sensors fixed on the motor.

[0091] S302: Use a multi-layer convolutional network to extract features from a discrete Gaussian distribution sequence to obtain a feature matrix, and straighten the feature matrix to obtain vibration features.

[0092] In an example, the process of feature extraction of discrete Gaussian distribution sequence using multi-layer convolutional network can be expressed as:

[0093]

[0094] In the formula, represents the discrete Gaussian distribution sequence elements, Represents the first elements, represents the convolution kernel weights, is the length of the discrete Gaussian distribution sequence, Indicates the size of the convolution kernel.

[0095] Assume that after a multi-layer convolutional network, a feature with a shape of (4, 32) can be obtained, that is, the feature matrix in this embodiment, 4 is the feature length, and 32 is the feature dimension, then it can be straightened into a 4×32=128-dimensional feature, that is, the vibration feature in this embodiment.

[0096] S303: Input the vibration features into the perception machine module for feature mapping to obtain the vibration frequency and vibration amplitude.

[0097] The vibration features are mapped to the frequency space and amplitude space respectively through the multi-layer perceptron module to obtain the vibration frequency and vibration amplitude. The above example is used to illustrate that the 128-dimensional features are mapped to the frequency space and amplitude space to obtain two one-dimensional outputs (vibration frequency and vibration amplitude).

[0098] It can be understood that the vibration features are extracted through a multi-layer convolutional network to obtain the vibration features, and then the vibration features are mapped through a multi-layer perceptron module to obtain the vibration frequency and vibration amplitude. This can extract effective information as much as possible and avoid noise interference in the calculation of the final vibration frequency and vibration amplitude, so as to improve the robustness and detection accuracy of vibration detection.

[0099] In one embodiment, feature points are extracted from each motor image in the motor image set to obtain a plurality of feature points corresponding to each motor image, including:

[0100] S1: Determine threshold parameters for each region in the motor image according to image parameters of each motor image in the motor image set.

[0101] In this step, the motor images in the motor image set can be analyzed to determine the overall image parameters of the motor images, such as brightness, texture, etc. Then, the threshold parameters of each area in the motor image can be determined according to the overall characteristics of the motor image.

[0102] Furthermore, the threshold parameters of each region of the motor image can be determined by a pre-trained neural network model, or by setting a mapping relationship between the overall characteristics of the motor and the threshold parameters, and then querying the mapping relationship to determine the threshold parameters of each region in the motor image. This application does not impose specific restrictions on this.

[0103] S2: Preprocess each motor image to highlight the main features of each motor image.

[0104] In this step, at least one processing including graying, normalization and filtering may be performed on each motor image, thereby eliminating noise and non-main features in each motor image to highlight the main features of each motor image.

[0105] S3: For each preprocessed motor image, traverse each pixel point of the motor image. If the absolute value of the difference between the grayscale value of the surrounding pixels of the pixel point and the grayscale value of the pixel point is less than the threshold parameter of the area where the pixel point is located, and the surrounding pixels are greater than the preset threshold, then the pixel point is determined as a feature point. After traversing the motor image, multiple feature points corresponding to the motor image are obtained.

[0106] In this embodiment, surrounding pixels refer to multiple pixels around the pixel. In other words, surrounding pixels may refer to all pixels around the pixel, or may refer to a portion of pixels around the pixel. The specific surrounding pixel determination rules may be determined based on comprehensive considerations of performance requirements and other aspects, and this application does not impose specific restrictions on this. It should be noted that the surrounding pixels here may refer to adjacent similar points, or may refer to pixels on the border of a geometric figure determined with the pixel as the center.

[0107] Exemplarily, when the pixel point being traversed is point A, assuming the preset threshold is 2, the surrounding pixel points of the pixel point are determined to be point B, point C and point D. At this time, the absolute value AB of the difference between the grayscale values ​​of point A and point B is calculated, as well as the absolute value AC of the difference between the grayscale values ​​of point A and point C, and the absolute value AD of the difference between the grayscale values ​​of point A and point D are calculated. If the absolute value AB, the absolute value AC and the absolute value AD are all smaller than the threshold parameter of the area where point A is located, it means that point A can be determined as a feature point. Otherwise, it means that point A cannot be determined as a feature point.

[0108] S4: After traversing each motor image, a plurality of feature points corresponding to each motor image are obtained.

[0109] It can be understood that the threshold parameters of different areas are determined based on the overall image parameters of the motor image, and then compared based on the threshold parameters of different areas when determining the feature points, so as to adapt to the motor images obtained from the videos to be detected shot in different environments, and further improve the robustness of motor vibration detection.

[0110] In one embodiment, determining a point pair set corresponding to the motor image set according to a plurality of feature points of adjacent motor images in the motor image set includes:

[0111] S1: For each motor image in the motor image set except the last frame of the motor image, each feature point corresponding to the motor image is matched with each feature point in the next motor image of the motor image to determine the feature point corresponding to each feature point of the motor image in the next motor image, and then form a point pair subset of the motor image and the next motor image.

[0112] The point pair subset includes multiple pairs of points, each pair of points includes two feature points. In an example, the point pair subset can be expressed as {[( ), ( )], [( ), ( )] …, [( ), ( ) ]},in, Indicates the frame number. Indicates For feature points, [( ), ( )] represents a pair of points, i.e. Frame motor image and The first For feature points.

[0113] In this step, for each motor image, a feature point matching method can be used to match each feature point corresponding to the motor image with each feature point in the next motor image of the motor image, and then after determining the feature point corresponding to each feature point of the motor image in the next motor image, a point pair subset of the motor image and the next motor image can be generated based on the feature point corresponding to each feature point of the motor image in the next motor image. Specifically, the feature point matching method can adopt the FLANN (Fast Library for Approximate Nearest Neighbors, nearest neighbor matching) method, or other applicable feature point matching methods, which are not specifically limited in this application.

[0114] S2: after determining the point pair subsets of each pair of adjacent motor images in the motor image set, generating a point pair set of the motor image set according to the point pair subsets of each pair of adjacent motor images in the motor image set.

[0115] It can be understood that in this embodiment, by matching the feature points of adjacent motor images, the offset of the motor vibration can be analyzed, and then the vibration frequency and vibration amplitude of the motor vibration can be determined.

[0116] In one embodiment, calculating the similarity of each pair of points in the point pair set includes:

[0117] S1: For each pair of points in the point pair set, determine the descriptor of the feature points in the pair of points.

[0118] S2: Calculate the Hamming distance between the feature points in the pair of points according to the descriptors of the feature points in the pair of points, and determine the similarity between the feature points in the pair of points according to the Hamming distance between the feature points in the pair of points.

[0119] Among them, the descriptor is a vector used to describe the local area information around the feature point. The descriptor can be generated when the feature point is determined. The Hamming distance refers to the number of different characters in the corresponding positions of two strings of equal length.

[0120] In this embodiment, when determining the descriptors of the feature points in the pair of points, the Hamming distance between the descriptors of the feature points in the pair of points is calculated, and the calculated Hamming distance is determined as the Hamming distance between the feature points in the pair of points, and then the similarity between the feature points in the pair of points is determined based on the Hamming distance between the feature points in the pair of points.

[0121] In one example, when determining the similarity between the feature points in the pair of points based on the Hamming distance between the feature points in the pair of points, the similarity between the feature points in the pair of points may be determined by querying a preset function model.

[0122] It can be understood that the smaller the Hamming distance between the descriptors of two feature points is, the more similar the descriptors of the two feature points are. This indicates that the matching degree between the feature points is high. By calculating the similarity between the feature points in each pair of points, the matching quality of each pair of points can be determined, and then screening can be performed based on the matching quality to improve the accuracy of the data for calculating the vibration frequency and vibration amplitude, thereby improving the accuracy of vibration detection of the motor.

[0123] In one embodiment, probability distribution modeling is performed on the feature point pair set to obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set, including:

[0124] S1: Calculate the offset value of each feature pair point in the feature point set.

[0125] In one example, the process of calculating the offset value of each feature pair point in the feature point pair set can be expressed as follows:

[0126]

[0127] In the formula, Indicates Frame motor image Offset value for feature pair points, ( ), ) indicates the The coordinates of the feature points in the feature pair. Indicates the frame number. Indicates Pair features with points.

[0128] S2: According to the offset value of each feature point pair in the feature point pair set, an offset sequence corresponding to the feature point pair set is formed.

[0129] In one example, the offset sequence can be expressed as: {( ), ( ),…, ( ) , …, ( )}, whose data shape is (L-1, N, 1), where L represents the number of motor images in the motor image set and N represents the number of feature points in the motor image. ( ) can represent The set of offsets in the frame motor image.

[0130] S3: quantize the offset sequence, and perform discrete Gaussian distribution modeling on the quantized offset sequence based on a preset Gaussian function to obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set.

[0131] In this step, the offset sequence is quantized according to a preset quantization grid size, and then discrete Gaussian distribution modeling is performed on the quantized offset sequence based on a preset Gaussian function to obtain a Gaussian distribution sequence corresponding to the feature point pair set.

[0132] In one example, the Gaussian function can be expressed as:

[0133]

[0134] In the formula, The offset sequence The Gaussian function value of the element, represents the standard deviation, Represents the mean.

[0135] Will Normalization is performed so that the sum of all calculated Gaussian function values ​​is 1. The normalization formula can be expressed as:

[0136]

[0137] In the formula, and Both represent element indexes. is the normalized offset sequence The Gaussian function value of each element. When each normalized Gaussian function value is determined, a discrete Gaussian distribution sequence can be determined based on this. At this time, the discrete Gaussian distribution sequence obtained is data with a shape of (L-1, N, 256). Then, the data of the second dimension and the third dimension in the discrete Gaussian distribution sequence are straightened to obtain data with a shape of (L-1, N*256). The straightened discrete Gaussian distribution sequence is used as the final discrete Gaussian distribution sequence. Finally, the discrete Gaussian distribution sequence is determined as the discrete Gaussian distribution sequence corresponding to the feature point pair set.

[0138] It can be understood that the one-dimensional distribution in the offset sequence is modeled into a discrete Gaussian distribution through discrete Gaussian distribution modeling, so as to facilitate input into the vibration detection model for feature extraction and mapping, thereby achieving high-precision motor vibration detection.

[0139] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0140] The motor vibration detection device provided in an embodiment of the present application is described below. The motor vibration detection device described below and the motor vibration detection method described above can be referenced to each other.

[0141] like Figure 4 As shown, the present application provides a motor vibration detection device 400, the device comprising:

[0142] The video acquisition module 401 is used to acquire the video to be detected;

[0143] An image segmentation module 402 is used to segment the motor object in each frame image in the video to be detected by using a preset video segmentation model to obtain a motor image set formed by the motor images corresponding to each frame image;

[0144] A feature point extraction module 403 is used to extract feature points from each motor image in the motor image set to obtain a plurality of feature points corresponding to each motor image, and determine a point pair set corresponding to the motor image set according to a plurality of feature points of adjacent motor images in the motor image set;

[0145] The similarity calculation module 404 is used to calculate the similarity of each pair of points in the point pair set, and select the N pairs of points with the highest similarity to form a feature point pair set;

[0146] The vibration detection module 405 is used to perform probability distribution modeling on the feature point pair set, obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set, extract vibration features from the Gaussian distribution sequence and map it to obtain vibration frequency and vibration amplitude.

[0147] In one embodiment, the image segmentation module includes:

[0148] A first model determination submodule, used to determine a preset video segmentation model;

[0149] The frame rate segmentation submodule is used to segment the video to be detected into picture frames according to the acquisition frame rate of the video to be detected, so as to obtain each frame image in the video to be detected;

[0150] An image segmentation submodule is used to obtain segmentation cue points, and input the segmentation cue points and each frame image into a video segmentation model, so as to segment the motor object in each frame image according to the segmentation cue points, and obtain a motor image corresponding to each frame image;

[0151] The set generation submodule is used to generate a motor image set based on each motor image.

[0152] In one embodiment, the vibration detection module includes:

[0153] The second model determination submodule is used to determine a preset vibration detection model; the vibration detection model includes a multi-layer convolutional network and a multi-layer perceptron module;

[0154] The feature extraction submodule is used to extract features from the Gaussian distribution sequence using a multi-layer convolutional network to obtain a feature matrix, and then straighten the feature matrix to obtain vibration features;

[0155] The feature mapping submodule is used to input the vibration features into the perception machine module for feature mapping to obtain the vibration frequency and vibration amplitude.

[0156] In one embodiment, the feature point extraction module includes:

[0157] A parameter determination submodule, used to determine the threshold parameters of each area in the motor image according to the image parameters of each motor image in the motor image set;

[0158] A preprocessing submodule, used for preprocessing each motor image to highlight the main features of each motor image;

[0159] A traversal submodule is used to traverse each pixel of each preprocessed motor image, and if the absolute value of the difference between the grayscale value of the surrounding pixels of the pixel and the pixel is less than the threshold parameter of the area where the pixel is located, and the surrounding pixels are greater than the preset threshold, the pixel is determined as a feature point, and after the motor image is traversed, a plurality of feature points corresponding to the motor image are obtained;

[0160] The feature point determination submodule is used to obtain a plurality of feature points corresponding to each motor image after traversing each motor image.

[0161] In one embodiment, the feature point extraction module includes:

[0162] A feature point matching submodule is used to match each feature point corresponding to each motor image except the last frame of the motor image in the motor image set with each feature point in the motor image next to the motor image, so as to determine the feature point corresponding to each feature point of the motor image in the next motor image, and then form a point pair subset of the motor image and the next motor image; the point pair subset includes a plurality of pairs of points, each pair of points includes two feature points;

[0163] The point pair set determination submodule is used to generate a point pair set of the motor image set according to the point pair subset of each pair of adjacent motor images in the motor image set after determining the point pair subset of each pair of adjacent motor images in the motor image set.

[0164] In one embodiment, the similarity calculation module includes:

[0165] A descriptor determination submodule is used to determine the descriptor of the feature point in each pair of points in the point pair set;

[0166] The similarity determination submodule is used to calculate the Hamming distance between the feature points in the pair of points according to the descriptors of the feature points in the pair of points, and determine the similarity between the feature points in the pair of points according to the Hamming distance between the feature points in the pair of points.

[0167] In one embodiment, the vibration detection module includes:

[0168] The offset value calculation submodule is used to calculate the offset value of each feature pair point in the feature point pair set;

[0169] A sequence determination submodule, used to form an offset sequence corresponding to the feature point pair set according to the offset value of each feature point pair set;

[0170] The sequence modeling submodule is used to quantize the offset sequence and perform discrete Gaussian distribution modeling on the quantized offset sequence based on a preset Gaussian function to obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set.

[0171] The division of each module in the above-mentioned motor vibration detection device is only used for illustration. In other embodiments, the motor vibration detection device can be divided into different modules as needed to complete all or part of the functions of the above-mentioned motor vibration detection device. Each module in the above-mentioned motor vibration detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0172] In one embodiment, the present application further provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the motor vibration detection method as described in any one of the above embodiments.

[0173] In one embodiment, the present application further provides a computer device having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the motor vibration detection method as described in any one of the above embodiments.

[0174] Indicatively, Figure 5 As shown, Figure 5 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 500 may be provided as a server. Figure 5 The computer device 500 includes a processing component 502, which further includes one or more processors, and a memory resource represented by a memory 501, for storing instructions executable by the processing component 502, such as an application. The application stored in the memory 501 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 502 is configured to execute instructions to perform the motor vibration detection method of any of the above embodiments.

[0175] The computer device 500 may further include a power supply component 503 configured to perform power management of the computer device 500, a wired or wireless network interface 504 configured to connect the computer device 500 to a network, and an input / output (I / O) interface 505. The computer device 500 may operate based on an operating system stored in the memory 501, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0176] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0177] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish an entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only include those elements, but also include other elements that are not clearly listed, or also include elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. Herein, the singular "one", "one" and "described / the" may also include plural forms, unless the context clearly indicates another way. It should also be understood that the terms "include / comprise" or "have" etc. specify the existence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the existence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof. At the same time, the term "and / or" used in this specification includes any and all combinations of the relevant listed items.

[0178] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0179] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A motor vibration detection method, characterized in that: The method comprises: Get the video to be detected; Segmenting the motor object in each frame image in the video to be detected by using a preset video segmentation model to obtain a motor image set formed by the motor images corresponding to each frame image; Extracting feature points from each motor image in the motor image set to obtain a plurality of feature points corresponding to each motor image, and determining a point pair set corresponding to the motor image set according to a plurality of feature points of adjacent motor images in the motor image set; Calculate the similarity of each pair of points in the point pair set respectively, and select the N pairs of points with the highest similarity to form a feature point pair set; Probability distribution modeling is performed on the feature point pair set to obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set, and then vibration features are extracted from the discrete Gaussian distribution sequence and mapped to obtain vibration frequency and vibration amplitude.

2. The motor vibration detection method according to claim 1, characterized in that: The method of using a preset video segmentation model to segment the motor object in each frame image in the video to be detected to obtain a motor image set formed by motor images corresponding to each frame image includes: Determine a preset video segmentation model; Dividing the video to be detected into picture frames according to the acquisition frame rate of the video to be detected, so as to obtain each frame image in the video to be detected; Acquire segmentation cue points, and input the segmentation cue points and each frame image into the video segmentation model, so as to segment the motor object in each frame image according to the segmentation cue points, and obtain the motor image corresponding to each frame image; A motor image set is generated based on the individual motor images.

3. The motor vibration detection method according to claim 1, characterized in that: The step of extracting vibration features from the Gaussian distribution sequence and mapping the vibration frequency and vibration amplitude includes: Determine a preset vibration detection model; the vibration detection model includes a multi-layer convolutional network and a multi-layer perceptron module; Using the multi-layer convolutional network to extract features from the Gaussian distribution sequence to obtain a feature matrix, and straightening the feature matrix to obtain vibration features; The vibration characteristics are input into the sensor module for feature mapping to obtain the vibration frequency and vibration amplitude.

4. The motor vibration detection method according to claim 1, characterized in that: The extracting feature points of each motor image in the motor image set to obtain a plurality of feature points corresponding to each motor image includes: Determine a threshold parameter for each area in the motor image according to an image parameter of each motor image in the motor image set; Preprocessing each motor image to highlight the main features of each motor image; For each preprocessed motor image, traverse each pixel of the motor image, if the absolute value of the difference between the grayscale value of the surrounding pixels of the pixel and the pixel is less than the threshold parameter of the area where the pixel is located, and the surrounding pixels are greater than the preset threshold, then the pixel is determined as a feature point, and after traversing the motor image, multiple feature points corresponding to the motor image are obtained; After traversing each motor image, multiple feature points corresponding to each motor image are obtained.

5. The motor vibration detection method according to claim 1, characterized in that: The step of determining a point pair set corresponding to the motor image set according to a plurality of feature points of adjacent motor images in the motor image set comprises: For each motor image except the last motor image in the motor image set, each feature point corresponding to the motor image is matched with each feature point in the next motor image of the motor image, so as to determine the feature point corresponding to each feature point corresponding to the motor image in the next motor image, and then form a point pair subset of the motor image and the next motor image; the point pair subset includes a plurality of pairs of points, each pair of points includes two feature points; After determining the point pair subsets of each pair of adjacent motor images in the motor image set, a point pair set of the motor image set is generated according to the point pair subsets of each pair of adjacent motor images in the motor image set.

6. The motor vibration detection method according to claim 1, characterized in that: The respectively calculating the similarity of each pair of points in the point pair set comprises: For each pair of points in the point pair set, determining a descriptor of a feature point in the pair of points; The Hamming distance between the feature points in the pair of points is calculated according to the descriptors of the feature points in the pair of points, and the similarity between the feature points in the pair of points is determined according to the Hamming distance between the feature points in the pair of points.

7. The motor vibration detection method according to claim 1, characterized in that: The performing probability distribution modeling on the feature point pair set to obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set includes: Calculate the offset value of each feature pair point in the feature point pair set; According to the offset value of each feature point pair in the feature point pair set, an offset sequence corresponding to the feature point pair set is formed; The offset sequence is quantized, and discrete Gaussian distribution modeling is performed on the quantized offset sequence based on a preset Gaussian function to obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set.

8. A motor vibration detection device, characterized in that: The device comprises: A video acquisition module is used to acquire the video to be detected; An image segmentation module is used to segment the motor object in each frame image in the video to be detected by using a preset video segmentation model to obtain a motor image set formed by the motor images corresponding to each frame image; A feature point extraction module, used to extract feature points from each motor image in the motor image set to obtain a plurality of feature points corresponding to each motor image, and determine a point pair set corresponding to the motor image set according to a plurality of feature points of adjacent motor images in the motor image set; A similarity calculation module is used to calculate the similarity of each pair of points in the point pair set, and select the N pairs of points with the highest similarity to form a feature point pair set; The vibration detection module is used to perform probability distribution modeling on the feature point pair set, obtain a discrete Gaussian distribution sequence corresponding to the feature point pair set, extract vibration features from the discrete Gaussian distribution sequence and map it to obtain vibration frequency and vibration amplitude.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the motor vibration detection method according to any one of claims 1 to 7.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the motor vibration detection method according to any one of claims 1 to 7 are performed.

Images