A semantic segmentation method and visual positioning method for vibration images

Abstract

The present invention discloses a semantic segmentation method and a visual positioning method for vibration images. The segmentation method includes: obtaining a vibration image; obtaining a target segmentation mask of the vibration image through a trained semantic segmentation network model. The vibration visual positioning method includes: using a vibration displacement extraction branch to calculate the zero-order moment and the first-order moment of the target segmentation mask of the vibration image; according to the calculated zero-order moment and first-order moment, obtaining the center point coordinates of each frame of the vibration image in the video, and then performing displacement offset regression on the center point coordinates of all video frames in turn, thereby obtaining target displacement data. The present invention uses a high-speed industrial camera as an image acquisition medium and a rotating body as an object of vibration displacement measurement, introduces a semantic segmentation method based on deep learning into the field of visual vibration measurement of rotating bodies, and verifies the feasibility of deep learning methods in visual vibration measurement of rotating bodies from multiple angles, and has the advantages of non-contact, long distance, and easy installation.

Description

Technical Field

[0001] The invention relates to a semantic segmentation method and a visual positioning method for vibration images, and belongs to the field of computer vision. Background Art

[0002] In practical applications, rotating bodies such as rotors, bearings, and spindles experience surface and internal wear over long periods of use, causing periodic vibration throughout the entire device. This can negatively impact the lifespan of precision components and machining accuracy, and can even lead to serious safety incidents. Therefore, accurately regressing the vibration displacement curve of a rotating body and determining the fault type of that structure is crucial for measuring the vibration displacement of rotating structures in complex scenarios.

[0003] With the rise of deep learning in the field of machine vision in recent years, scholars have begun to introduce it into the field of visual vibration measurement because it can obtain more image features that are not obvious through learning from a large amount of data. When performing vibration displacement measurement based on the deep learning method of target detection of Bounding Box, the target position is first predicted by the detection network and marked with Bounding Box. Then, the coordinates of the target center point are calculated based on the predicted coordinates of the upper left and lower right corners of the Bounding Box. Finally, the vibration displacement curve is obtained by regressing the coordinates of the target center point in each frame of the image. Figure 2 As can be seen from the coordinate extraction diagram of the target detection method, its accuracy depends entirely on the fit between the Bounding Box and the target being detected. When detecting rotating targets, it is often as follows Figure 2 (a) and Figure 2 (b) The Bounding Box is larger or smaller than the target. Figure 2 As shown in (c), the Bounding Box fits the target perfectly. Zooming in reveals that there's still a slight error between the predicted target center coordinates A1 and the true coordinates A0. This can cause a slight shift in the relative offset of the target's center vibration point between frames, leading to significant errors in the final regressed displacement curve. Summary of the Invention

[0004] The present invention provides a semantic segmentation method for vibration images, which segments targets in vibration images using a constructed CSP+Mobiledets backbone network, and further uses the segmented images to obtain target center points, thereby achieving the acquisition of displacement curves.

[0005] The technical solution of the present invention is:

[0006] According to another aspect of the present invention, a semantic segmentation method for vibration images is provided, comprising: obtaining a vibration image; obtaining a target segmentation mask of the vibration image using a trained semantic segmentation network model; the semantic segmentation network model uses a segmentation network U-Net as a basic framework, uses a CSP+Mobiledets backbone network as an encoder branch to perform preliminary feature extraction, and then uses a decoder branch to enhance feature extraction of the extracted five feature layers, thereby achieving feature layer decoding of the segmentation network.

[0007] The semantic segmentation network model adds the Dice-loss loss function when training the model.

[0008] The CSP+Mobiledets backbone network built uses the FIBN module and TC module in the Mobiledets network, replaces the CBR module with the CBM module, and uses the CSP module in CPSNet to fuse the semantic information of different depths.

[0009] The CSP+Mobiledets backbone network is used as the Encoder branch for preliminary feature extraction, including five feature extraction modules connected in sequence; the first feature extraction module includes a CBM module, a TC module, and a CSP1 module, and the result obtained by performing a CBM module convolution on the input image is fused through the CSP1 module and the TC module once to obtain the first output feature F1; the second feature extraction module includes a CBM module, a FIBN module, a TC module, and a CSP2 module, and the result obtained by performing two CBM module convolutions on the input image is fused with the result after convolutions of the CSP2 module, the FIBN module with an 8-channel expansion ratio, and the TC module three times to obtain the second output feature F2; the third feature extraction module includes a CBM module, a FIBN module, and a CSP3 module, and the result obtained by performing two CBM convolutions on the input image is The CSP3 module is respectively fused with the convolution results of the FIBN module with an 8-channel expansion ratio five times and the FIBN module with a 4-channel expansion ratio three times to obtain the third output feature F3; the fourth feature extraction module includes the CBM module, the FIBN module, and the CSP4 module. The input image is convolved with the CBM twice to obtain the results, and the CSP4 module is convolved with the FIBN module with an 8-channel expansion ratio once, the FIBN module with a 4-channel expansion ratio three times, and the FIBN module with an 8-channel expansion ratio four times to obtain the fourth output feature F4; the fifth feature extraction module includes the CBM module, the FIBN module, and the CSP5 module. The input image is convolved with the CBM twice to obtain the results, and the CSP5 module is convolved with the FIBN module with an 8-channel expansion ratio four times. Finally, a CBM convolution is performed on the output image to obtain the fifth output feature.

[0010] The FIBN module fuses the first 1×1 convolution and the subsequent 3×3 depth-separable convolution in the IBN module into a single 3×3 regular convolution, and the expansion multiple of the intermediate feature layer channel is determined by the number expansion factor s>1 of the 3×3 regular convolution kernel.

[0011] The TC module introduces two compression factors e1 and e2 in the first 1×1 convolution and the subsequent 3×3 convolution.

[0012] According to another aspect of the present invention, a method for visually locating the center point of a target in a vibration image is also provided, comprising: using a vibration displacement extraction branch to calculate the zero-order moment and the first-order moment of the target segmentation mask of the vibration image obtained by the semantic segmentation method for vibration images described in claim 1; obtaining the center point coordinates of each frame of the vibration image in the video based on the calculated zero-order moment and first-order moment, and then performing displacement offset regression on the center point coordinates of all video frames in turn, thereby obtaining target displacement data.

[0013] The coordinates of the center point of the vibration image are calculated using the formula Calculation; where m 10 and m 01 is the first-order moment in the x and y directions, m 00 It is the zero-order moment of the entire target area.

[0014] The beneficial effects of the present invention are:

[0015] The present invention uses a high-speed industrial camera as an image acquisition medium and a rotating body such as a rotor in a high-speed video as the object of vibration displacement measurement. It introduces a semantic segmentation method based on deep learning into the field of visual vibration measurement of rotating bodies and verifies the feasibility of the deep learning method in visual vibration measurement of rotating bodies from multiple perspectives. The method has the advantages of non-contact, long-distance, and easy installation. Specifically, the present invention uses the classic segmentation network U-Net as the basic framework and makes extensive use of the FIBN and TC convolutional layers in Mobiledets to construct the CSP+Mobiledets backbone network of the present invention. The network is used as the Encoder branch for preliminary feature extraction, and the CSP (Cross stage The partial) module fuses the semantic information of different depths and replaces the common convolution module CBR (Conv+BN+Relu) with CBM (Conv+BN+Mish), which can make the network achieve better accuracy and generalization. Furthermore, the five effective feature layers F1, F2, F3, F4 and F5 extracted from the Encoder branch are then enhanced with feature extraction by the Decoder branch to achieve feature layer decoding of the segmentation network, and each pixel is classified to obtain the final segmentation mask. Secondly, the coordinates of the center point of the image are extracted frame by frame through the vibration displacement extraction branch, so as to more accurately measure the vibration displacement offset. On the basis of the above, in order to solve the problem that the binary cross entropy of the original U-Net network model can easily cause the model to converge to the local optimal solution, the Dice-loss loss function is added on the basis of the original binary cross entropy to calculate the final loss of the network model, which is more conducive to solving the imbalance problem of positive and negative samples in complex backgrounds; further, the present invention uses the synchronously collected eddy current displacement signal as the standard displacement offset, and compares the visual displacement measurement results under the same speed. It can be seen from the time-frequency domain diagram and the axis trajectory diagram that the vibration displacement curve regressed by the semantic segmentation network has a high degree of fit with the displacement signal obtained by the eddy current sensor, which further verifies the reliability of the method of the present invention in vibration displacement measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a structural block diagram of the present invention;

[0017] Figure 2 Extract schematic diagram for detection method;

[0018] Figure 3 is a rotating object image;

[0019] Figure 4 This is the architecture diagram of the U-Net segmentation network model;

[0020] Figure 5This is a diagram of the CSP+Mobiledets backbone network architecture of the present invention;

[0021] Figure 6 Comparison diagram of the convolutional module structures of IBN, FIBN, and TC;

[0022] Figure 7 Flowchart for training model;

[0023] Figure 8 Segmentation mask for feature decoding;

[0024] Figure 9 Image schematic diagram for calculating the center point coordinates for the rotation body displacement extraction branch;

[0025] Figure 10 This is a time domain diagram comparing the semantic segmentation algorithm and the eddy current sensor signal (abscissa - time, ordinate - amplitude);

[0026] Figure 11 The frequency domain diagram (abscissa - frequency, ordinate - amplitude) comparing the semantic segmentation algorithm and the eddy current sensor signal.

[0027] Figure 12 Axis trajectory diagram comparing semantic segmentation algorithm and eddy current sensor signal. DETAILED DESCRIPTION

[0028] The invention will be further described below with reference to the accompanying drawings and embodiments, but the content of the present invention is not limited to the scope of the drawings.

[0029] Example 1: Figure 1-12 As shown, according to one aspect of an embodiment of the present invention, a semantic segmentation method for vibration images is provided, including: obtaining a vibration image; obtaining a target segmentation mask of the vibration image through a trained semantic segmentation network model; the semantic segmentation network model uses the segmentation network U-Net as a basic framework, uses the CSP+Mobiledets backbone network as the Encoder branch to perform preliminary feature extraction, and then uses the Decoder branch to enhance feature extraction of the extracted 5 feature layers, so as to achieve feature layer decoding of the segmentation network.

[0030] Furthermore, the semantic segmentation network model adds a Dice-loss loss function when training the model.

[0031] Furthermore, the CSP+Mobiledets backbone network was built, which continued to use the FIBN module and TC module in the Mobiledets network, replaced the CBR module with the CBM module, and used the CSP module in CPSNet to fuse the semantic information of different depths.

[0032] Furthermore, the CSP+Mobiledets backbone network is used as the Encoder branch for preliminary feature extraction, including five feature extraction modules connected in sequence; the first feature extraction module includes a CBM module, a TC module, and a CSP1 module, and the input image is convolved with the CBM module once, and the result obtained is fused through the CSP1 module and the TC module once to obtain the first output feature F1; the second feature extraction module includes a CBM module, a FIBN module, a TC module, and a CSP2 module, and the input image is convolved with the CBM module twice, and the result obtained is fused with the CSP2 module, the FIBN module with an 8-channel expansion ratio, and the TC module three times to obtain the second output feature F2; the third feature extraction module includes a CBM module, a FIBN module, and a CSP3 module, and the input image is convolved with the CBM module twice to obtain the second output feature F2. The results are fused using the CSP3 module with five convolutions of the FIBN module with an 8-channel expansion ratio and three convolutions of the FIBN module with a 4-channel expansion ratio to obtain the third output feature F3; the fourth feature extraction module includes the CBM module, the FIBN module, and the CSP4 module. The input image is convolved with CBM twice to obtain the results, and the CSP4 module is convolved with the FIBN module with an 8-channel expansion ratio, the FIBN module with a 4-channel expansion ratio, and the FIBN module with an 8-channel expansion ratio four times to obtain the fourth output feature F4; the fifth feature extraction module includes the CBM module, the FIBN module, and the CSP5 module. The input image is convolved with CBM twice to obtain the results, and the CSP5 module is convolved with the FIBN module with an 8-channel expansion ratio four times to obtain the fifth output feature.

[0033] Furthermore, the FIBN module fuses the first 1×1 convolution and the subsequent 3×3 depth-separable convolution in the IBN module into a single 3×3 regular convolution, and the expansion multiple of the intermediate feature layer channel is determined by the number expansion factor s>1 of the 3×3 regular convolution kernel.

[0034] Furthermore, the TC module introduces two compression factors e1 and e2 in the first 1×1 convolution and the subsequent 3×3 convolution.

[0035] The U-Net segmentation network model architecture is as follows Figure 4As shown. The left half is the five effective feature layers F1, F2, F3, F4 and F5 extracted from the Encoder branch, and the right half is the Decoder branch used to enhance feature extraction. The specific implementation method is: first upsample the F5 feature layer, and then splice and fuse it with F4 to obtain the fused feature E4 (32×32×512); secondly, upsample the E4 feature layer, and then splice and fuse it with F3 to obtain the fused feature E3 (64×64×256); then upsample the E3 feature layer, and then splice and fuse it with F2 to obtain the fused feature layer E2 (128×128×128); then upsample the E2 feature layer, and then splice and fuse it with F1 to obtain the fused feature layer E1 (256×256×64). In this way, the feature layer decoding of the segmentation network is achieved, and finally the final segmentation mask is obtained by classifying each pixel. In this paper, the segmentation network U-Net is used as the basic framework, and the CSP+Mobiledets backbone network is used as the Encoder branch for preliminary feature extraction. Figure 5 As shown in Figure 1, specifically, first input an image of size 512×512×3, perform a CBM module convolution, and fuse the result obtained by the CSP1 module and the TC module once to obtain an image of size 256×256×32, and then extract this image layer as the F1 feature layer; the image of size 256×256×32 is convolved twice by the CBM module, and then the result obtained by convolving the CSP2 module with an 8-channel expansion ratio FIBN module and three TC modules is fused to obtain an image of size 128×128×32, and then extract this image as the F2 feature layer; similarly, the image of size 128×128×32 is convolved twice by the CBM module, and then the result obtained by convolving the CSP3 module with five 8-channel expansion ratios FIBN modules and three 4-channel expansion ratios is fused. The convolution results of the FIBN module of the example are fused to obtain an image of size 64×64×64, and this image layer is regarded as the F3 feature layer; secondly, the 64×64×64 image is convolved twice by CBM to obtain the result, and then the CSP4 module is fused with the convolution results of the FIBN module with an 8-channel expansion ratio, the FIBN module with a 4-channel expansion ratio three times, and the FIBN module with an 8-channel expansion ratio four times to obtain an image of size 32×32×128, and this image layer is used as the F4 feature layer; finally, the 32×32×128 image is convolved twice by CBM, and the convolution result is fused using the CSP5 module and the FIBN module with a 8-channel expansion ratio four times to obtain an image of size 16×16×256, and finally, a CBM convolution is performed again to output the image, which is regarded as the F5 feature layer.

[0036] The CSP+Mobiledets backbone network framework is shown in Table 1: Layer represents the convolution operation, Output size represents the feature map size output after the convolution operation, s represents the ratio of channel expansion in the FIBN convolution operation, e1 represents the ratio of the first channel compression in the TC convolution operation, e2 represents the ratio of the second channel compression in the TC convolution operation, and n represents the number of convolution operations. Taking the segmentation network U-Net as the basic framework, the CSP+Mobiledets backbone network is built. First, when designing the feature extraction backbone network, the FIBN (Fused inverted bottleneck layer) and TC (Tucker convolution layers) in the Mobiledets network are extensively used. The common convolution module CBR (Conv+BN+Relu) is replaced by CBM (Conv+BN+Mish), and the CSP (Cross stage partial) module in CPSNet is used to fuse the semantic information of different depths. Among them, the Fused inverted bottleneck layer (FIBN) and Tucker convolution layers (TC) can be regarded as variants of the IBN layer. The specific structures of the IBN layer, FIBN layer and TC layer are as follows: Figure 6 (a), 6(b) and Figure 6 (c) The key to the IBN layer is the depth-wise separable convolution, which uses a combination of 3×3 depth-wise separable convolution and 1×1 convolution to replace the complex conventional convolution operation, as shown in Figure 6 As shown in (a). The most significant change in the FIBN layer is the fusion of the first 1×1 convolution and the subsequent 3×3 depthwise separable convolution in the IBN layer into a single 3×3 regular convolution. The overall structure still belongs to the anti-bottleneck layer. The expansion multiple of the intermediate feature layer channels is determined by the expansion factor s>1 of the number of 3×3 regular convolution kernels. The TC layer introduces two compression factors, e1 and e2, in the first 1×1 convolution and the subsequent 3×3 convolution, with their values ​​generally being 0.25 and 0.75. This constructed CSP+Mobiledets backbone network as a feature extraction method can greatly improve the accuracy of the segmentation model with almost no loss in computational speed, ensuring excellent performance in vibration displacement measurement.

[0037] Table 1

[0038] Layer Outputsize s e1 e2 n CBM 256×256×32 - - - 1 CSP1 256×256×16 - - - 1 TC 256×256×16 - 0.25 0.75 1 Concat 256×256×32 - - - 1 CBM 256×256×16 - - - 1 CBM 128×128×32 - - - 1 CSP2 128×128×16 - - - 1 FIBN 128×128×16 8 - - 1 TC 128×128×16 - 0.25 0.25 3 Concat 128×128×32 - - - 1 CBM 128×128×32 - - - 1 CBM 64×64×64 - - - 1 CSP3 64×64×32 - - - 1 FIBN 64×64×32 8 - - 5 FIBN 64×64×32 4 - - 3 Concat 64×64×64 - - - 1 CBM 64×64×64 - - - 1 CBM 32×32×128 - - - 1 CSP4 32×32×64 - - - 1 FIBN 32×32×64 8 - - 1 FIBN 32×32×64 4 - - 3 FIBN 32×32×64 8 - - 4 Concat 32×32×128 - - - 1 CBM 32×32×128 - - - 1 CBM 16×16×256 - - - 1 CSP5 16×16×128 - - - 1 FIBN 16×16×128 4 - - 4 Concat 16×16×256 - - - 1 CBM 16×16×256 - - - 1

[0039] The acquired image data was annotated using the annotation tool Labelme, dividing it into training, validation, and test datasets. The annotation boxes are uniformly sized circular boxes. For the dataset division, we conducted comparative experiments using datasets with different training ratios, as shown in Table 2. 6-4 indicates a random split of the dataset into 60% training and 40% validation; 7-3 indicates a random split of the dataset into 70% training and 30% validation; 8-2 indicates a random split of the dataset into 80% training and 20% validation; and 9-1 indicates a random split of the dataset into 90% training and 10% validation. Precision (P), recall (R), intersection over union (IOU), and F1 (F1) were used as model evaluation metrics. Table 2 shows the comparison results of P, R, IOU, and F1 for datasets with different training ratios. The boldfaced columns in Table 2 indicate the best performance in the corresponding metric.

[0040] Table 2

[0041] DataSets 6-4 7-3 8-2 9-1 P(％) 98.94 97.46 99.60 96.69 R(％) 98.71 99.18 99.26 98.90 IOU (%) 97.68 96.69 98.87 97.50 F1 0.9882 0.9832 0.9943 0.9873

[0042] From the experimental results in Table 2, it can be seen that when 80% of the entire dataset is used as a training dataset and 20% is used as a validation dataset, the model's P, R, IOU, and F1 values ​​are the highest, and the network model can achieve the best performance at this time. Therefore, the training dataset for all network models in this application is 480 images, the validation dataset is 120 images, the test dataset is 100 images, the batch size is set to 4, and the number of training times is uniformly set to 200 epochs. All experiments were conducted on a desktop computer (GPU is NVIDIA GeForce RTX 3060, video memory is 12G, CPU is Intel Core i9-10850K, memory is 32G), and all deep learning algorithms are configured with a unified operating environment (windows 10, Cuda 11.0, Pytorch 1.7.1, torchvision 0.8.2).

[0043] Use the training data set to train the deep learning network model, and then use the test data set to evaluate the model to obtain the optimal weight file, specifically: See the attached Figure 7Before formal training, configure the training hyperparameters in the train.py file of the semantic segmentation network model. The hyperparameters of the configuration file mainly include the number of extracted images, learning rate, optimizer, momentum, number of iterations, and weight decay coefficient. The rest of the parameters are the default values. In this embodiment, the batch_size of the number of images extracted at a time is 4, the learning rate is 1e-4, the optimizer is Adam, the momentum is 0.9, the number of iterations is 200, and the weight decay coefficient is 0. The rest of the hyperparameters are the default values. The deep learning semantic segmentation network model is trained. The training object is mainly the target segmentation mask. The training results are presented in the form of each pass. The weight file is obtained after one training iteration; the train.py file in the deep learning semantic segmentation network model is called to start training. After reaching the set number of training times, multiple candidate weight files will be obtained; the obtained candidate weight files are evaluated using the test data set in the vibration displacement data set. If the matching degree is low compared with the evaluation model, they are directly discarded. If the matching degree is high compared with the evaluation model, the semantic segmentation network model with the best effect is screened out, and its weight file is set as the optimal weight file; the obtained optimal weight file is loaded into the semantic segmentation network, and then all network parameters are frozen and locked to obtain a frozen model, which is also the model used for detection. In order to verify the feasibility of the innovative points of the segmentation network of the present invention, the present invention selects the original U-Net algorithm, Mobiledets-U-Net (U-Net with the backbone network replaced by Mobiledets), WCSP-Ours (the algorithm of this paper does not use the CSP feature fusion module in the backbone network), Relu-Ours (the algorithm of this paper with the Relu activation function in the backbone network), and CEloss-Ours (the algorithm of this paper using only the cross entropy loss) for experimental comparison with the algorithm of the present invention, and uses "precision P, recall rate R, intersection-over-union ratio IOU, F1 value, model parameter number W, RMSE-X and RMSE-Y as model evaluation indicators" to obtain the experimental results shown in Table 3. The bold marks in Table 3 indicate that it has the best performance in the corresponding indicators.

[0044] Table 3

[0045] Algorithm U-Net Mobiledets-U-Net WCSP-Ours Relu-Ours CELoss-Ours Ours P(％) 92.54 98.54 98.97 98.50 97.37 99.60 R(％) 96.74 99.15 98.92 98.66 98.52 99.26 IOU (%) 89.74 97.72 97.92 97.20 95.97 98.87 F1 0.9459 0.9885 0.9895 0.9858 0.9794 0.9943 W(MB) 35 19 14 14 14 14 RMSE-X 0.4206 0.3887 0.3388 0.5397 0.3406 0.1219 RMSE-Y 0.7103 0.8183 0.2508 0.4849 0.4900 0.0958

[0046] As can be seen from Table 3, whether it is P, R, IOU, F1, or RMSE in both directions, the algorithm of the present invention that integrates all innovations is superior to others. When the Encoder branch of U-Net is replaced with the Mobiledets backbone network, the overall performance of the algorithm decreases slightly regardless of whether the CSP module is used or the Relu activation function is replaced by the Mish activation function. When the network only uses cross entropy loss, the average performance also decreases significantly.

[0047] In order to verify the segmentation performance of the algorithm of the present invention, we selected FCN, PSPNet, DeepLabV3+, LR-ASPP, HRNet and the algorithm of the present invention for quantitative and qualitative comparison in terms of evaluation indicators and segmentation mask effects. The comparison results of P, R, IOU, F1, W, FPS, Time, RMSE-X and RMSE-Y of different segmentation algorithms are shown in Table 4. The bold marks in the table indicate that they have the best performance in the corresponding indicators.

[0048] Table 4

[0049] Algorithm FCN PSPNet DeepLabV3+ LR-ASPP HRNet Ours P(％) 93.28 99.44 97.15 95.95 99.51 99.60 R(％) 93.57 99.09 97.57 92.66 99.25 99.26 IOU (%) 87.65 98.55 94.84 89.15 98.77 98.87 F1 0.9342 0.9927 0.9735 0.9426 0.9938 0.9943 W(MB) 33 47 55 3 30 14 FPS 2.34 22.31 16.42 7.25 12.16 18.45 Time(s) 0.4273 0.0448 0.0609 0.1379 0.0822 0.0542 RMSE-X 0.9269 0.1594 0.6106 0.8637 0.1919 0.1219 RMSE-Y 0.7047 0.1444 0.5349 0.6510 0.1516 0.0958

[0050] As can be seen from Table 4, PSPNet, HRNet, and the segmentation algorithm of the present invention are very close to and superior to the other algorithms in terms of P, R, IOU, and F1. However, HRNet's larger number of model parameters results in slower computation speed, and its RMSE in both the X and Y directions is much higher than that of the algorithm of the present invention. Although PSPNet has a slight advantage over the algorithm of the present invention in computation time, its accuracy is more important. The accuracy of the segmentation algorithm determines the accuracy of the final vibration displacement to a certain extent, as evidenced by the fact that the RMSE of the algorithm of the present invention in both directions is much lower than that of PSPNet. From the perspective of model complexity, the number of model parameters W, computation speed FPS, and computation time Time are all important factors, but the relationship between model parameter number and computation time is not directly proportional. Due to the extensive use of depthwise separable convolution in LR-ASPP, although its parameter number is far smaller than that of the algorithm of the present invention, its computation time is 0.0837s longer. More importantly, the RMSE of FCN, DeepLabV3+, and LR-ASPP is several times that of the algorithm of the present invention. These three algorithms suffer from excessive errors when measuring the vibration displacement of rotating objects. From the perspective of segmentation evaluation indicators, model complexity and measurement error, the algorithm of the present invention is undoubtedly a better choice.

[0051] According to another aspect of an embodiment of the present invention, a method for visually locating the center point of an object in a vibration image is provided, comprising:

[0052] The vibration displacement extraction branch (canny edge detection algorithm) is used to calculate the zero-order moment and first-order moment of the target segmentation mask of the vibration image obtained by the semantic segmentation method for vibration images. The above semantic segmentation method for vibration images can be used to input a newly captured vibration image sequence to be detected into the frozen network model for detection, and output a target segmentation mask with strong contrast frame by frame; Figure 8 As shown in the figure, a target segmentation mask of a vibration image is shown. The network sets the target pixel to 255 and the background pixel to 0. The image principle of the vibration displacement extraction branch to calculate the center point coordinates is as follows Figure 9 As shown;

[0053] Based on the calculated zero-order moment and first-order moment, the center point coordinates of each vibration image frame in the video are obtained, and then the displacement offset regression is performed on the center point coordinates of all video frames in turn to obtain the target displacement data.

[0054] Furthermore, the center coordinates of the vibration image are calculated using the formula Calculation; where m 10 and m 01 is the first-order moment in the x and y directions, m 00 It is the zero-order moment of the entire target area (i.e. m 00 Represents the total number of pixels in the image that are 255, i.e. the sum of the mask areas; m 10 Represents the sum of the x-coordinates of the pixels in the image, which is 255; m 01 Represents the pixel value in the image as 255, which is the sum of the y coordinates of the mask area.

[0055] Since the final output of the target detection is framed by the Bounding Box, this detection-based method cannot guarantee a high degree of fit between the Bounding Box and the target, and therefore will inevitably cause a large displacement error when regressing the target center point coordinates. The segmentation network classifies each pixel point, which can achieve pixel-level accuracy, and the error in the final calculation of the target center point coordinates is relatively smaller. The specific verification is as follows: Use a high-speed industrial camera and an eddy current sensor to synchronously collect the image signal and voltage signal of the rotating body, and then use the signal collected by the high-speed industrial camera as image data and the signal collected by the eddy current sensor as eddy current data for experimental comparison; use the above method to obtain displacement data, normalize the displacement data of the rotating body to obtain the vibration displacement curve of the rotating body, and use the displacement signal synchronously collected by the eddy current sensor as the standard displacement offset, respectively, from the vibration displacement time domain diagram, frequency domain diagram and axis trajectory in the X direction. Figure 3 First, the time domain diagram of the vibration displacement curve of the rotating body is obtained, and the time domain diagram is fast Fourier transformed to obtain the frequency domain diagram, and then compared with the standard displacement curve of the eddy current signal, such as Figure 10 The figure shows a time domain comparison of the semantic segmentation algorithm and the eddy current sensor signal in a unified coordinate system. In general, both can fit the eddy current signal, but the peak value of the algorithm displacement curve is slightly smaller than the eddy current signal. Considering the different sensitivity of the camera and the eddy current sensor to the signal, this phenomenon that some time domain signals cannot be fully fitted is within an acceptable error range. Figure 11The comparison of the semantic segmentation algorithm and the frequency domain diagram of the eddy current sensor signal in the unified coordinate system also shows that the segmentation algorithm of the present invention can highly fit the eddy current signal at more peaks, and it is found that the image segmentation network algorithm of the present invention has a high degree of fit with it; at the same time, the axis trajectory diagram that can be regressed from the vibration displacement curve is compared and it is found that the axis trajectory noise of the semantic image segmentation network algorithm of the present invention is the least, and it is more fitting to the axis trajectory diagram of the eddy current signal, such as Figure 12 As shown in the figure, the comparison between the time-frequency diagram and the axis trajectory diagram shows that the algorithm of the present invention benefits from the pixel-level accuracy of the segmentation algorithm, so its vibration displacement measurement effect far exceeds that of the target detection algorithm. This further confirms the superior performance of the segmentation algorithm used in the present invention in the vibration displacement measurement of rotating bodies.

[0056] The specific embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A semantic segmentation method for vibration images, characterized in that: include: Acquire vibration images; The target segmentation mask of the vibration image is obtained through the trained semantic segmentation network model. The semantic segmentation network model uses the segmentation network U-Net as the basic framework, uses the CSP+Mobiledets backbone network as the encoder branch for preliminary feature extraction, and then uses the decoder branch to upsample the extracted five feature layers layer by layer and splice them with the encoder features to achieve feature layer decoding of the segmentation network. The semantic segmentation network model adds a Dice-loss loss function when training the model; The CSP+Mobiledets backbone network we built uses the FIBN and TC modules in the Mobiledets network, replaces the CBR module with the CBM module, and uses the CSP module in CPSNet to fuse features of semantic information at different depths. The CSP+Mobiledets backbone network is used as the Encoder branch for preliminary feature extraction, including five feature extraction modules connected in sequence; The first feature extraction module includes a CBM module, a TC module, and a CSP1 module. The input image is convolved with the CBM module once, and the result obtained is fused through the CSP1 module and the TC module once to obtain the first output feature F1. The second feature extraction module includes a CBM module, a FIBN module, a TC module, and a CSP2 module. The input image is convolved with the CBM module twice, and then the result obtained by convolving with the CSP2 module, the FIBN module with an 8-channel expansion ratio, and the TC module three times is fused to obtain the second output feature F2. The third feature extraction module includes a CBM module, a FIBN module, and a CSP3 module. The input image is convolved twice with the CBM, and the results are fused using the CSP3 module, five convolutions with the FIBN module at an 8-channel expansion ratio, and three convolutions with the FIBN module at a 4-channel expansion ratio to obtain the third output feature F3. The fourth feature extraction module includes a CBM module, a FIBN module, and a CSP4 module. The input image is convolved twice with the CBM, and the results are fused using the CSP4 module, a FIBN module with an 8-channel expansion ratio, a FIBN module with a 4-channel expansion ratio, and a FIBN module with an 8-channel expansion ratio four times, to obtain the fourth output feature F4. The fifth feature extraction module includes a CBM module, a FIBN module, and a CSP5 module. The input image is convolved twice with the CBM, and the results are fused using the CSP5 module and four convolutions with the FIBN module at an 8-channel expansion ratio. Finally, the output image is convolved with the CBM module again to obtain the fifth output feature. The FIBN module fuses the first 1×1 convolution and the subsequent 3×3 depth-separable convolution in the IBN module into a single 3×3 regular convolution, and the expansion multiple of the intermediate feature layer channel is determined by the number expansion factor s>1 of the 3×3 regular convolution kernel.

2. The semantic segmentation method for vibration images according to claim 1, characterized in that The TC module introduces two compression factors e1 and e2 in the first 1×1 convolution and the subsequent 3×3 convolution.

3. A method for visually locating the center point of a vibration image target, characterized in that: include: Using the vibration displacement extraction branch, the zero-order moment and the first-order moment of the target segmentation mask of the vibration image obtained by the semantic segmentation method for vibration images according to claim 1 are calculated; Based on the calculated zero-order moment and first-order moment, the center point coordinates of each vibration image frame in the video are obtained, and then the displacement offset regression is performed on the center point coordinates of all video frames in turn to obtain the target displacement data.

4. The method for visually locating the center point of a vibration image target according to claim 3, characterized in that: The coordinates of the center point of the vibration image are calculated using the formula Calculation; where m 10 and m 01 is the first-order moment in the x and y directions, m 00 It is the zero-order moment of the entire target area.

Images