Visual vibration measurement method and system suitable for rotating body inclination visual angle mapping

Abstract

The invention discloses a visual vibration measurement method and system suitable for rotator inclination visual angle mapping, and the method comprises the steps: carrying out the camera external parameter calibration of rotor vibration image data at m different visual angles theta at random, and obtaining an included angle theta'between the optical center of a camera and a rotating shaft of a rotor vibration test bench, and a projection coefficient; building a semantic segmentation algorithm network model of multi-resolution feature fusion; loading a semantic segmentation algorithm network model of multi-resolution feature fusion corresponding to the optimal weight as a frozen model; inputting the newly obtained vibration video / test set of the to-be-detected rotor into the frozen model for detection, and outputting a mask graph frame by frame; extracting the horizontal coordinate and the vertical coordinate of each pixel of the target segmentation mask part of the mask image frame by frame, and extracting the center point coordinate of the target segmentation mask part according to the displacement calculation parameter of the rotating body; and connecting the central point coordinates of the target segmentation mask parts of all the images in the vibration video of the rotor to be detected as a real vibration displacement curve of the rotating body. Visual measurement of the vibration displacement of the rotating body under different visual angles can be realized.

Description

Technical Field

[0001] The present invention relates to a visual vibration measurement method and system applicable to the inclined view mapping of a rotating body, belonging to the fields of visual vibration displacement measurement and computer vision. Background Art

[0002] In practical applications, rotating bodies such as rotors, bearings, and spindles will experience wear on the surface and internal structures during long-term use, which will also cause periodic vibrations of the entire equipment. This will inevitably affect the service life of precision components and the accuracy of machining, and even lead to extremely serious safety accidents. Therefore, it is of great significance to accurately regress the vibration displacement curve of the rotating body for measuring the vibration displacement of rotating structural bodies in complex scenarios.

[0003] In traditional vibration displacement measurement and detection methods, contact and non-contact sensors are usually used to measure the vibration of rotating bodies. Contact sensors cannot collect accurate vibration signals well for low-frequency vibrations; non-contact sensors are limited by the measurement range and installation position, and have weak anti-interference ability, and there are also certain defects in vibration displacement measurement. Compared with traditional sensor measurement methods, visual sensors can be arranged at a distance and can also collect vibration displacement images from more positions, but the fixed angle limits the vibration displacement accuracy in the horizontal direction of two-dimensional visual vibration measurement algorithms. Currently, most visual sensors for collecting vibration images of rotating bodies collect images from one or two vibration directions, and do not fully reflect the flexible arrangement and real-time measurement characteristics of visual sensors at any angle. Summary of the Invention

[0004] The present invention provides a visual vibration measurement method and system applicable to the inclined view mapping of a rotating body, which is used to realize the visual measurement of the vibration displacement of a rotating body at different viewing angles through a semantic segmentation algorithm network model constructed by multi-resolution feature fusion.

[0005] The technical solution of the present invention is as follows:

[0006] According to the first aspect of the present invention, a visual vibration measurement method applicable to the inclined view mapping of a rotating body is provided, including the following steps:

[0007] Step 1: Use an image acquisition device to acquire rotor vibration image data at m different viewing angles θ;

[0008] Step 2: Randomly select one image from the rotor vibration image data at m different viewing angles θ for camera extrinsic parameter calibration to obtain the angle θ' between the camera optical center and the axis of the rotor vibration test bench and the projection coefficient;

[0009] Step 3: Use a labeling tool to label the targets in the rotor vibration images taken from m different perspectives to obtain class labels, and divide the labeled rotor vibration images into a training set and a test set;

[0010] Step 4: Build a semantic segmentation algorithm network model with multi-resolution feature fusion;

[0011] Step 5: Use the training set to train the semantic segmentation algorithm network model with multi-resolution feature fusion, and then use the test set to evaluate the model to obtain the optimal weight file;

[0012] Step 6: Load the semantic segmentation algorithm network model corresponding to the optimal weights as a frozen model;

[0013] Step 7: Input the newly obtained rotor vibration video / test set to be detected into the frozen model for detection, and output the mask map frame by frame;

[0014] Step 8: Extract the abscissa and ordinate of each pixel of the target segmentation mask part frame by frame for the mask map, and extract the center point coordinates of the target segmentation mask part according to the rotating body displacement calculation parameters in Step 2; Connect the center point coordinates of the target segmentation mask parts of all images in the rotor vibration video to be detected as the true vibration displacement curve of the rotating body.

[0015] Furthermore, use the frozen model in the visual vibration measurement method applicable to the mapping of the inclined perspective of the rotating body for the detection of rotor vibration images at "35° ≤ viewing angle θ ≤ 90°".

[0016] Furthermore, when using the image acquisition device to collect rotor vibration image data at m different viewing angles θ, synchronously collect the corresponding eddy current data as the standard signal through an eddy current sensor.

[0017] Furthermore, the semantic segmentation algorithm network model for multi-resolution feature fusion includes a feature extraction backbone network, a multi-resolution module, and a decoder connected in sequence; the feature extraction backbone network is composed of a C3BR module, a C4BR module, and a CPCA module connected in sequence; the multi-resolution module divides the input into four features with resolutions from high to low and sends them into the first-stage basic module layer respectively. The output of any basic module layer in the first stage is added to the output after alignment with the outputs of the remaining basic module layers in the first stage through upsampling and downsampling operations as the input of the corresponding basic module layer in the second stage. The output of any basic module layer in the second stage is added to the output after alignment with the outputs of the remaining basic module layers in the second stage through upsampling and downsampling operations as the input of the corresponding basic module layer in the third stage. The output of any basic module layer in the third stage is added to the output after alignment with the outputs of the remaining basic module layers in the third stage through upsampling and downsampling operations and then spliced through an upsampling operation to obtain the output; the decoder is composed of a C3BR, a 3×3 convolution, and a nearest neighbor upsampling connected in series in sequence.

[0018] Furthermore, when training the semantic segmentation algorithm network model for multi-resolution feature fusion, the Focal-loss and Focal-Tversky-loss loss functions are introduced and added to the original MSE-loss as the total loss function.

[0019] According to the second aspect of the present invention, there is provided a visual vibration measurement system applicable to the mapping of the tilted perspective of a rotating body, including the module of the visual vibration measurement method applicable to the mapping of the tilted perspective of a rotating body described in any one of the above.

[0020] According to the third aspect of the present invention, there is provided a processor, and the processor is used to execute operations, and the operations include executing the visual vibration measurement method applicable to the mapping of the tilted perspective of a rotating body described in any one of the above.

[0021] The beneficial effects of the present invention are:

[0022] The present invention uses a high-speed industrial camera as the image acquisition medium and a rotating body such as a rotor in a high-speed video as the object for measuring vibration displacement, and proposes an "angle-category" mapping method suitable for measuring the tilt angle of a rotating body, attempting to achieve vibration displacement measurement in two directions, namely in-plane and out-of-plane. Moreover, this measurement method has advantages such as non-contact, long-distance, and easy installation. Specifically, the present invention designs a feature extraction backbone composed of a C3BR module, a C4BR module, and CPCA for preliminary feature extraction. Among them, the C3BR module and the C4BR module are used to increase the receptive field of the network for the rotor image and downsample the rotor image, and CPCA is used to extract the detailed information of the rotor image. Further, a multi-resolution module is used for enhanced feature extraction to achieve the feature layer decoding of the segmentation network, and each pixel point is classified to obtain the final segmentation mask. Secondly, the coordinates of the center point of the image are extracted frame by frame to more accurately measure the vibration displacement offset of the rotating body. On this basis, the Focal-loss and Focal-Tversky-loss functions are added to the original binary cross-entropy to calculate the final loss of the network model, which is more conducive to solving the problem of imbalance between positive and negative samples in a complex background. Furthermore, the present invention uses the synchronously collected eddy current displacement signal as the standard displacement offset, compares the visual displacement measurement results at the same rotational speed, and it can be seen from the time-frequency domain diagram and the shaft center orbit diagram that the vibration displacement curve regressed by the semantic segmentation network has a high degree of fitting with the displacement signal obtained by the eddy current sensor, further verifying the reliability of the method of the present invention in measuring the vibration displacement of a rotating body. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is the flow chart of the method of the present invention;

[0024] Figure 2 is the schematic diagram of the acquisition by the high-speed industrial camera at a preset angle;

[0025] Figure 3 are the target images at three different angles;

[0026] Figure 4 is the picture of the calibration plate placement;

[0027] Figure 5 is the flow chart for constructing datasets at different angles;

[0028] Figure 6 is the block diagram of the semantic segmentation algorithm network model with multi-resolution feature fusion proposed by the research method of the present invention;

[0029] Figure 7 is the structural block diagram of the feature extraction backbone network;

[0030] Figure 8It is a structural block diagram of the modules included in the semantic segmentation algorithm network model for multi-resolution feature fusion;

[0031] Figure 9 It is a structural block diagram of the multi-resolution module;

[0032] Figure 10 It is a structural block diagram of the decoder;

[0033] Figure 11 It is a flow chart of the training model;

[0034] Figure 12 It is an effect diagram of placing the mask graph with reduced transparency output by the model on the original image;

[0035] Figure 13 It is a time-domain graph (abscissa - time, ordinate - amplitude) for comparing the signals of the present invention and the eddy current sensor;

[0036] Figure 14 It is an axis orbit graph for comparing the signals of the present invention and the eddy current sensor. Detailed implementation manners

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention. It should be noted that, without conflict, the embodiments and features in the embodiments of this application can be combined with each other arbitrarily.

[0038] Embodiment 1: As Figures 1 - 14 shown, according to the first aspect of the embodiments of the present invention, a visual vibration measurement method applicable to the tilt view mapping of a rotating body is provided, including the following:

[0039] Step 1: Use an image acquisition device and an eddy current sensor to respectively collect rotor vibration image data and corresponding eddy current data at m different viewing angles θ; where, 35° ≤ viewing angle θ ≤ 90°;

[0040] Exemplarily: Use a high-speed industrial camera and an eddy current sensor to synchronously collect the image data and voltage signals of the rotor. The schematic diagram of the collection by the high-speed industrial camera at a preset viewing angle is as Figure 2 shown, Figure 3The rotor vibration images at 45°, 60°, and 75° angles of view collected by the high-speed industrial camera are further shown, and the voltage signal collected by the eddy current sensor is used as eddy current data for experimental comparison. Among them, both the rotor vibration image data and the eddy current data are synchronously collected on a high-speed rotor vibration test bench (Nanjing Southeast University Z-03). The parameters of the rotor being collected are a radius of 32.75 mm and a mass of 590 g. Two sensors with different acquisition principles, namely a high-speed industrial camera (Thousand-Eye Wolf M220, set with a resolution of 1024×1024 and a frame rate of 2000 Hz) and an eddy current sensor (Shanghai Oudu-9000, set with a Scale Factory of 2.5 V / mm and a signal sampling rate of 2000 Hz), are used as measuring devices for collecting the displacement of the rotating body. At the same time, it is set that the high-speed industrial camera and the photographic lamp (Jinbei EF-200LED) are at the same horizontal height as the rotor. The power of the photographic lamp is set to 200 W and the color temperature is 5500 K to increase the contrast of the picture and highlight the target rotor. The image data at the three angles of view consists of 6000-frame sequences of images at the same rotation speed for 3 seconds each.

[0041] Step 2: Randomly select one image from the rotor vibration image data at m different angles of view θ for camera extrinsic parameter calibration to obtain the angle θ' between the camera optical center and the axis of the rotor vibration test bench and the projection coefficient; compare the current angle of view θ with the corresponding angle θ'. If the absolute value of the difference between the two is within the error range, then use the angle θ' between the camera optical center and the axis of the rotor vibration test bench and the projection coefficient as the calculation parameters for the displacement of the rotating body in the rotor vibration image at the current angle of view θ to calculate the displacement of the rotating body; otherwise, if the absolute value of the difference between the two is not within the error range, it indicates that the calculation result of the displacement of the rotating body in the rotor vibration image at this angle of view does not meet the preset performance indicators.

[0042] Exemplarily, when the angle of view θ takes values in [35°, 60°], the absolute value of the difference between θ and θ' being less than 5° indicates being within the error range; when the angle of view θ takes values in (60°, 70°], the absolute value of the difference between θ and θ' being less than 1.5° indicates being within the error range; when the angle of view θ takes values in (70°, 90°], the absolute value of the difference between θ and θ' being less than 0.5° indicates being within the error range.

[0043] In specific applications, first perform camera extrinsic parameter calibration on the image through matlab to obtain the angle between the camera optical center and the axis of the rotor vibration test bench and the projection coefficient. The placement of the calibration board is as Figure 4 shown. The calibration board is parallelly attached to the axis of the rotor in the rotor vibration test bench. After camera calibration, the angle between the camera optical center and the axis of the rotor vibration test bench can be obtained as shown in Table 1. It can be seen from Table 1 that the errors between 45° and 60° and their corresponding θ' are all within the error range.

[0044] Table 1 Camera Calibration Results

[0045] θ θ' Projection coefficient 1 / cosθ 1 / cosθ' 45° 42.16° 0.08 mm / pixel 1.41 1.34 60° 59.85° 0.08 mm / pixel 2.00 1.99 75° 76.52° 0.08 mm / pixel 3.86 4.28

[0046] Step 3: Using an annotation tool, annotate the targets in the rotor vibration images taken from m different perspectives to obtain class labels, and divide the rotor vibration images with labels into a training set and a test set;

[0047] Exemplarily, use the annotation tool Labelme to perform class annotation on the smooth circumferential surface projection at the front end of the rotor in rotor vibration images of different classes with respective color pixels. For example, Figure 5 as shown (First class: The smooth circumferential surface projection at the front end of the rotor at a 45° perspective is annotated with (255, 0, 0); Second class: The smooth circumferential surface projection at the front end of the rotor at a 60° perspective is annotated with (0, 255, 0); Third class: The smooth circumferential surface projection at the front end of the rotor at a 75° perspective is annotated with (0, 255, 255)). Divide the annotated data set into a 90% training set and a 10% test set, with 540 images in the training set and 60 images in the test set.

[0048] Step 4: Build a semantic segmentation algorithm network model with multi-resolution feature fusion;

[0049] As Figure 6 shown, the semantic segmentation algorithm network model with multi-resolution feature fusion includes a feature extraction backbone network, a multi-resolution module, and a decoder connected in sequence; the feature extraction backbone network consists of a C3BR module, a C4BR module, and a CPCA module connected in sequence; as Figure 7 、 9 shown, the multi-resolution module divides the input into four features with resolutions from high to low and sends them into the first-stage basic module layer respectively. The output of any basic module layer in the first stage is added to the output of the remaining basic module layers in the first stage after alignment through upsampling and downsampling operations as the input of the corresponding basic module layer in the second stage. The output of any basic module layer in the second stage is added to the output of the remaining basic module layers in the second stage after alignment through upsampling and downsampling operations as the input of the corresponding basic module layer in the third stage. The output of any basic module layer in the third stage is added to the output of the remaining basic module layers in the third stage after alignment through upsampling and downsampling operations and then spliced through upsampling operation to obtain the output; as Figure 10 shown, the decoder consists of a C3BR, a 3×3 convolution, and a nearest neighbor upsampling connected in series in sequence.

[0050] Exemplarily, refer to Figure 9, the basic module layers of each stage of the multi-resolution module are all four. Taking the input process from the basic module layer of the first stage to the basic module layer of the second stage as an example: the multi-resolution module divides the input into four features with resolutions from high to low, as Figure 9 shown, the resolution decreases from top to bottom. The outputs of the first basic module layer of the first stage, the output of the second basic module layer of the first stage after double upsampling, the output of the third basic module layer of the first stage after quadruple upsampling, and the output of the fourth basic module layer of the first stage after octuple upsampling are added together as the input of the first basic module layer of the second stage; the output of the first basic module layer of the first stage after double downsampling, the output of the second basic module layer of the first stage, the output of the third basic module layer of the first stage after double upsampling, and the output of the fourth basic module layer of the first stage after quadruple upsampling are added together as the input of the second basic module layer of the second stage; the output of the first basic module layer of the first stage after quadruple downsampling, the output of the second basic module layer of the first stage after double downsampling, the output of the third basic module layer of the first stage, and the output of the fourth basic module layer of the first stage after double upsampling are added together as the input of the third basic module layer of the second stage; the output of the first basic module layer of the first stage after octuple downsampling, the output of the second basic module layer of the first stage after quadruple downsampling, the output of the third basic module layer of the first stage after double downsampling, and the output of the fourth basic module layer of the first stage are added together as the input of the fourth basic module layer of the second stage; the upsampling and downsampling processes from the second stage to the third stage are the same as those of the first and second stages; the outputs of the first basic module layer of the third stage, the output of the second basic module layer of the third stage after double upsampling, the output of the third basic module layer of the third stage after quadruple upsampling, and the output of the fourth basic module layer of the third stage after octuple upsampling are added together as "Feature 1 obtained by adding the output of the first basic module layer of the third stage and the outputs of the remaining basic module layers of the third stage aligned through upsampling and downsampling operations", and thus Features 2, 3, and 4 are obtained (the acquisition of Features 2, 3, and 4 is the same as the input of the second, third, and fourth basic module layers of the second stage), and the outputs of the operations of doubling the upsampling of Feature 1, quadrupling the upsampling of Feature 2, and octupling the upsampling of Feature 3 and Feature 4 are concatenated.

[0051] Furthermore, the basic module layer adopts Basic Block×4 in HR-Net.

[0052] In the above, the feature extraction backbone network is composed of a C3BR module, a C4BR module, and a CPCA module connected in sequence. As Figure 8 shown, the C3BR module is: 3×3Conv+BatchNorm+Relu, the C4BR module is 4×4Conv+BatchNorm+Relu, and the CPCA module is a module well-known in the art.

[0053] Exemplarily: taking the rotor vibration image as the input, the main features of the rotor image are extracted in the feature extraction backbone network. Among them, it goes through the C3BR module, where the number of channels changes from 3 channels to 64 channels, and the image size remains unchanged. Then it passes through the C4BR module, where the number of channels changes from 64 channels to 256 channels, and the image size becomes one-fourth of the original. Then it goes through the CPCA module, which includes a channel attention part and a spatial attention part. The channel attention part is used to obtain the features in the channel dimension. The spatial attention part uses depthwise separable convolutions, that is, different combinations of depthwise separable convolutions are used to replace complex conventional convolution operations. As Figure 8 shown, the 1×n type convolution is used to learn the features of the rotor image from the height dimension, and the n×1 type convolution is used to learn the features of the rotor image from the width dimension. By adjusting the shape of the convolution kernel, the extraction accuracy of the network for the features of the rotor image in different dimensions is improved. Therefore, the CPCA module pays attention to the detailed features of the rotor image from different perspectives in the channel dimension, height dimension, and width dimension, reducing the training difficulty of the network and thus achieving better feature extraction performance. Then, the feature layers extracted are strengthened by the multi-resolution module. Finally, there is a decoding part. The decoder is composed of C3BR, 3×3 convolution, and nearest neighbor upsampling connected in series in sequence. C3BR is used to integrate the spliced feature maps to make the network more robust and expressive; the 3×3 convolution adjusts the number of channels of the feature map to the number of categories to match the subsequent classification requirements; the nearest neighbor upsampling restores the feature map to the size of the input image to ensure the extraction performance of the algorithm for the rotor displacement. Through the decoding part, the network can effectively capture the edge information of the rotor and generate high-quality segmentation results.

[0054] Furthermore, when training the model, the Focal-loss and Focal-Tversky-loss loss functions are introduced and added to the original MSE-loss as the total loss function. Through the improved loss function, the network can effectively distinguish the part of the rotor as the segmentation target from the part as the background, thereby improving the accuracy of the network in extracting the rotor displacement.

[0055] Step 5: Use the training set to train the semantic segmentation algorithm network model with multi-resolution feature fusion, and then use the test set to evaluate the model to obtain the optimal weight file;

[0056] Refer to Figure 11 and the specific content of the said Step 5 is as follows:

[0057] Step 5.1: Before starting the training, it is necessary to configure the training hyperparameters of the training script in the semantic segmentation algorithm network model. The main hyperparameters to be modified are as follows: the number of extracted images, the learning rate, the optimizer, the momentum, the number of iterations, and the weight decay coefficient. The rest of the parameters are default values. In this embodiment, the batch size of the number of images extracted each time is 2, the input image size is 1024×1024, the learning rate is 4e-3, the number of iterations is 300, and the learning rate decay coefficient is 1e-4. The rest of the hyperparameters are default values. The number of training rounds is 300 rounds. The environment configuration is a desktop computer (13th Gen Intel(R) Core(TM) i7-13700K, RAM 32G, GPU NVIDIA GeForce RTX 4080 16G), and the algorithm running environment is windows10, Cuda11.8, Pytorch2.1.1, torchvision0.16.1.

[0058] Step 5.2: Train the semantic segmentation algorithm network model. The form of the training result is the weight file obtained after each training iteration; after reaching the set number of training times, multiple candidate weight files are obtained.

[0059] Step 5.3: Use the test set in the rotating body vibration displacement dataset to evaluate the candidate weight files obtained in Step 5.2. If the matching degree is low compared with the evaluation model, it is directly discarded. If the matching degree is high compared with the evaluation model, the semantic segmentation network model with the best effect is selected, and its weight file is defined as the optimal weight file. The accuracy, recall rate, and intersection over union are used as evaluation indicators, and the intersection over union is the first evaluation indicator.

[0060] Further, the specific process of Step 5.2 is as follows:

[0061] Step 5.2.1: Call the training script to train the semantic segmentation algorithm network model;

[0062] Step 5.2.2: Randomly extract one batch_size of images from the training set as the current training samples;

[0063] Step 5.2.3: Put each image in the training samples in Step 5.2.2 into the semantic segmentation network model in turn to update the weight parameters; the update of the weight parameters is specifically as follows: randomly initialize the weight parameters or load the pre-trained weight parameters, perform the forward propagation calculation of the convolutional neural network to obtain a set of intermediate parameters, and then use the intermediate parameters to update the weight parameters through backpropagation; the new weight parameters will replace the old weight parameters used for the forward propagation calculation before.

[0064] Step 5.2.4: Denote the process of performing one forward and backward propagation on all images in the training set described in Step 5.2.2 as one training of the semantic segmentation network model, and save a weight file each time training is performed; repeat Step 5.3.2 to Step 5.3.3 until the number of times of training the network model reaches the set number of iterations;

[0065] Step 5.2.5: After reaching the set number of training times, obtain multiple candidate weight files.

[0066] Step 6: Load the semantic segmentation algorithm network model with multi-resolution feature fusion corresponding to the optimal weight as the frozen model.

[0067] Step 7: Input the newly obtained rotor vibration video to be detected / test set into the frozen model for detection, and output the mask map frame by frame; obtain the angle θ' between the optical center of the camera and the axis of the rotor vibration test bench of any image in the rotor vibration video to be detected and the projection coefficient as the calculation parameters of the rotational body displacement of all images in the rotor vibration video to be detected;

[0068] Specifically: Input the newly obtained rotor vibration video to be detected / test set into the frozen model for detection. If the detected target is the first category, the frozen model sets the target pixel value to (255, 0, 0); if it is the second category, the target 2 pixel value is set to (0, 255, 0); if it is the third category, the target pixel value is set to (0, 255, 255), and the background pixels are set to 0, and output a mask map with strong contrast frame by frame. Exemplarily, for the purpose of demonstration, the mask maps obtained by detecting one rotor vibration image of each of the three categories in the test set by the model of the present invention are placed on the original image with reduced transparency, and the effect is as Figure 12 shown. It can be seen from Figure 12 that the position of the target segmentation mask part in the mask map obtained by the model of the present invention matches the original image.

[0069] Step 8: Extract the abscissa and ordinate of each pixel of the target segmentation mask part of the mask map frame by frame, and extract the center point coordinates of the target segmentation mask part according to the rotational body displacement calculation parameters in Step 7; connect the center point coordinates of the target segmentation mask parts of all images in the rotor vibration video to be detected as the true vibration displacement curve of the rotational body;

[0070] The specific expression is as follows:

[0071]

[0072] where Y and Z are the center point coordinates of the target segmentation mask, that is, the position coordinates of the rotor in the true coordinate axes y and z directions; u i 、v iare the abscissa and ordinate of the i-th mask pixel in the target segmentation mask of the image in the rotor vibration video to be detected, m is the total number of generated mask pixels, K is the projection coefficient, and θ' is the angle between the camera optical center and the axis of the rotor vibration test bench.

[0073] Step 9: According to the test set, the displacement signal synchronously collected by the eddy current sensor is used as the standard displacement offset, and the rotor vibration displacement curve (ours) obtained by the algorithm of the present invention is compared and verified from the time-domain diagram, frequency-domain diagram, and shaft center locus of the vibration displacement in the X direction. Figure 3 Compare and verify from three angles. First, obtain the time-domain diagram of the vibration displacement curve of the rotating body, perform a fast Fourier transform on the time-domain diagram to obtain the frequency-domain diagram, and compare it with the standard displacement curve of the eddy current signal respectively. As Figure 13 shown in the comparison of the time-domain diagrams of the semantic segmentation algorithm and the eddy current sensor signal in the unified coordinate system. Generally, the trends can basically fit the eddy current signal, but the peak value of the algorithm displacement curve is slightly smaller than that of the eddy current signal. Considering the difference in the sensitivity of the camera and the eddy current sensor to the signal, this phenomenon that some time-domain signals cannot be completely fitted is within the acceptable error range. It can be seen that the algorithm of the present invention has a very high fitting degree with it; at the same time, by comparing the shaft center locus diagrams that can be regressed from the vibration displacement curves, it is found that the shaft center locus of the algorithm of the present invention has very little noise and is also more fitted to the shaft center locus diagram of the eddy current signal. As Figure 14As shown. Further, based on the test set, the algorithm proposed in the present invention demonstrated excellent performance in the comprehensive quantitative comparison of performance indicators with different deep neural networks (including PSPNet, DeepLabV3+, SegFormer, U-Net, HRNet, etc.), as shown in Table 2. From the comprehensive evaluation of most indicators such as precision (P), recall (R), and F1-score, the algorithm of the present invention is at the best performance level in each indicator. Especially in terms of F1-score and IoU, the algorithm of the present invention achieved the best indicators compared with other algorithms, indicating that the generated segmentation mask can more accurately fit the segmentation mask of the true annotation, thereby improving the accuracy of vibration measurement. In the evaluation result of RMSE (root mean square error), although the vibration displacements generated by all algorithms did not fully fit the standard signal collected by the eddy current sensor, the classification training method adopted in the present invention significantly improved the generalization ability of the algorithm and was able to effectively compare the results of the control group. In addition, although the algorithm improved the detection performance, it did not comprehensively consider the detection effects of different types and their different directions. However, when the included angle between the optical center of the camera and the axis of the rotor vibration test bench is 45° and 60° respectively, the vibration displacements extracted by the algorithm of the present invention performed excellently in the three best indicators, and at the 60° included angle, the vibration displacements in two different directions both reached the best indicators. By calculating the average value (MRMSE) of all RMSEs and conducting a comprehensive analysis, the results showed that the algorithm of the present invention reached the optimal level in the overall performance of extracting vibration displacements. From the perspective of the number of parameters (PARAMS) and computational complexity (GFLOPS) of the algorithm, the present invention made a lightweight improvement to HRNetV2, significantly reducing the number of parameters and computational complexity while improving the comprehensive performance of the algorithm. PSPNet has a lower number of parameters and computational complexity, but the comprehensive performance of the network is poor. U-Net has a larger number of parameters and computational complexity, but it did not achieve the best detection performance, indicating that a large number of parameters and computational complexity do not necessarily result in excellent network performance, while a smaller number of parameters and computational complexity, such as SegFormer and the algorithm proposed in this study, can also obtain good network performance. In summary, the algorithm proposed in the present invention demonstrated excellent comprehensive performance in terms of performance, accuracy, and efficiency (PARAMS and GFLOPS), and was able to effectively optimize the computational resources of the algorithm, with broad application prospects.

[0074] Table 2 Comprehensive performance comparison of different networks, bold indicates the best indicators

[0075] algorithm PSPNet DeepLabV3+ SegFormer U-Net HRNetV2 The present invention 45°-P 99.08 91.88 98.91 99.18 99.12 99.73 45°-R 99.4 99.98 98.79 99.48 99.7 99.25 45°-F1 99.24 95.76 98.85 99.33 99.41 99.49 45°-IOU 98.49 91.87 97.72 98.67 98.83 98.98 60°-P 99.37 94.97 98.79 99.28 99.45 99.66 60°-R 99.28 99.84 98.63 99.54 99.53 99.4 60°-F1 99.32 97.34 98.71 99.41 99.14 99.53 60°-IOU 98.66 94.82 97.45 98.83 98.99 99.07 75°-P 99.03 96.82 98.58 98.57 98.85 98.95 75°-R 98.72 99.13 96.32 99.03 98.86 98.88 75°-F1 98.87 97.96 97.44 98.80 98.85 98.91 75°-IOU 97.78 96.01 95.0 97.63 97.74 97.85 45°-Y-RMSE 0.0281 0.1125 0.0349 0.0348 0.0273 0.0204 45°-Z-RMSE 0.0452 0.1723 0.0341 0.0265 0.0454 0.0272 60°-Y-RMSE 0.0546 0.1264 0.0547 0.0653 0.0452 0.0282 60°-Z-RMSE 0.0520 0.1152 0.0420 0.0510 0.0315 0.0225 75°-Y-RMSE 0.0992 0.1087 0.1289 0.0770 0.0956 0.0964 75°-Z-RMSE 0.0461 0.0623 0.0553 0.0453 0.0438 0.0536 MRMSE 0.0542 0.1162 0.0583 0.0500 0.0481 0.0414 PARAMS (MB) 2.376 5.814 3.715 24.891 9.637 7.798 GFLOPS (GB) 24.117 211.534 54.126 1807 149.348 128.740

[0076] According to the second aspect of the embodiments of the present invention, a visual vibration measurement system applicable to the mapping of the tilt perspective of a rotating body is provided, including the module of the visual vibration measurement method applicable to the mapping of the tilt perspective of a rotating body described in any one of the above. Specifically, it includes: a first module for performing step 1: collecting rotor vibration image data at m different perspectives θ using an image acquisition device; a second module for performing step 2: randomly selecting one image from the rotor vibration image data at m different perspectives θ for camera extrinsic parameter calibration to obtain the angle θ' between the camera optical center and the axis of the rotor vibration test bench and the projection coefficient; a third module for performing step 3: labeling the targets in the rotor vibration images taken at m different perspectives through a labeling tool to obtain class labels, and dividing the rotor vibration images with labels into a training set and a test set; a fourth module for performing step 4: building a semantic segmentation algorithm network model with multi-resolution feature fusion; a fifth module for performing step 5: training the semantic segmentation algorithm network model with multi-resolution feature fusion using the training set, and then evaluating the model using the test set to obtain the optimal weight file; a sixth module for performing step 6: loading the semantic segmentation algorithm network model with multi-resolution feature fusion corresponding to the optimal weight as a frozen model; a seventh module for performing step 7: inputting the newly obtained rotor vibration video / test set to be detected into the frozen model for detection, and outputting a mask image frame by frame; an eighth module for performing step 8: extracting the abscissa and ordinate of each pixel of the target segmentation mask part of the mask image frame by frame, and extracting the center point coordinates of the target segmentation mask part according to the rotating body displacement calculation parameters in step 2; connecting the center point coordinates of the target segmentation mask parts of all images in the rotor vibration video to be detected as the true vibration displacement curve of the rotating body. As used above, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the above embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated. For the parts not detailed in each module, reference can be made to the relevant descriptions of this embodiment.

[0077] According to the third aspect of the embodiments of the present invention, a processor is provided, and the processor is used to perform operations, and the operations include performing the visual vibration measurement method applicable to the mapping of the tilt perspective of a rotating body described in any one of the above.

[0078] The specific implementation manners of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above implementation manners. Within the knowledge scope of those of ordinary skill in the art, various changes can be made without departing from the purpose of the present invention.

Claims

1. A visual vibration measurement method applicable to the inclined view mapping of a rotating body, characterized in that, It includes the following steps: Step 1: Use an image acquisition device to collect rotor vibration image data at m different viewing angles θ; Step 2: Randomly select one image from the rotor vibration image data at m different viewing angles θ for camera extrinsic parameter calibration to obtain the angle θ' between the camera optical center and the axis of the rotor vibration test bench and the projection coefficient; Step 3: Use a labeling tool to label the targets in the rotor vibration images taken at m different viewing angles to obtain class labels, and divide the labeled rotor vibration images into a training set and a test set; Step 4: Build a semantic segmentation algorithm network model with multi-resolution feature fusion; Step 5: Use the training set to train the semantic segmentation algorithm network model with multi-resolution feature fusion, and then use the test set for model evaluation to obtain the optimal weight file; Step 6: Load the semantic segmentation algorithm network model with multi-resolution feature fusion corresponding to the optimal weights as a frozen model; Step 7: Input the newly obtained rotor vibration video / test set to be detected into the frozen model for detection, and output the mask map frame by frame; Step 8: Extract the abscissa and ordinate of each pixel of the target segmentation mask part of the mask map frame by frame, and extract the center point coordinates of the target segmentation mask part according to the rotating body displacement calculation parameters in Step 2; Connect the center point coordinates of the target segmentation mask parts of all images in the rotor vibration video to be detected as the true vibration displacement curve of the rotating body.

2. The visual vibration measurement method applicable to the tilt perspective mapping of a rotating body according to claim 1, characterized in that Use the frozen model in the visual vibration measurement method applicable to the mapping of the inclined viewing angle of the rotating body for the detection of rotor vibration images under "35° ≤ viewing angle θ ≤ 90°".

3. The visual vibration measurement method applicable to the tilt perspective mapping of a rotating body according to claim 1, characterized in that, When using an image acquisition device to collect rotor vibration image data at m different viewing angles θ, synchronously collect the corresponding eddy current data as the standard signal through an eddy current sensor.

4. The visual vibration measurement method applicable to the perspective mapping of the tilted rotating body according to claim 1, characterized in that The semantic segmentation algorithm network model with multi-resolution feature fusion includes a feature extraction backbone network, a multi-resolution module, and a decoder connected in sequence; the feature extraction backbone network is composed of a C3BR module, a C4BR module, and a CPCA module connected in sequence; the multi-resolution module divides the input into four features with resolutions from high to low and sends them into the first-stage basic module layer respectively. The output of any basic module layer in the first stage is added to the output after alignment with the outputs of the remaining basic module layers in the first stage through upsampling and downsampling operations as the input of the corresponding basic module layer in the second stage. The output of any basic module layer in the second stage is added to the output after alignment with the outputs of the remaining basic module layers in the second stage through upsampling and downsampling operations as the input of the corresponding basic module layer in the third stage. The output of any basic module layer in the third stage is added to the output after alignment with the outputs of the remaining basic module layers in the third stage through upsampling and downsampling operations and then spliced through upsampling operations to obtain the output; the decoder is composed of C3BR, 3×3 convolution, and nearest neighbor upsampling connected in series in sequence.

5. The visual vibration measurement method applicable to the tilt perspective mapping of a rotating body according to claim 1, characterized in that, When training the semantic segmentation algorithm network model with multi-resolution feature fusion, the Focal-loss and Focal-Tversky-loss loss functions are introduced and added to the original MSE-loss as the total loss function.

6. A visual vibration measurement system applicable to the tilt-angle mapping of a rotating body, characterized in that, A module including the visual vibration measurement method applicable to the tilt perspective mapping of a rotating body according to any one of claims 1-5.

7. A processor, characterized in that, The processor is used to perform operations, and the operations include performing the visual vibration measurement method applicable to the tilt perspective mapping of a rotating body according to any one of claims 1-5.

Images