Image deblurring network model and visual measurement method for vibration displacement of rotating body

Abstract

The application discloses an image deblurring network model and a visual measurement method for vibration displacement of a rotating body, and the model comprises a shallow feature extraction module, a deep feature extraction module, a feature fusion module, a multi-scale feature extraction module, a dense residual module, an RDM module and an attention mechanism.The deblurring network model is used in the data collection stage of visual vibration measurement, so that the image data definition and feature enhancement of the image data are improved, and the performance requirements such as frame rate and resolution of a camera required for collecting the image data are effectively reduced; furthermore, by comparing a displacement signal diagram generated by Ground-Truth image data, a displacement signal diagram generated by image data processed by the application and a displacement signal diagram generated by blurred image data, the practical engineering application value of the application is proved.

Description

TECHNICAL FIELD

[0001] The present application relates to an image deblurring network model and a visual measurement method for vibration displacement of a rotating body, and belongs to the field of artificial intelligence image deblurring reconstruction and visual measurement. BACKGROUND

[0002] The visual-based fault diagnosis method has attracted more and more attention of scholars due to its advantages of long distance, non-contact, multi-point detection and the like. However, in the actual visual measurement engineering application, the image displacement and strain field measurement has obvious defects in the image acquisition, processing and displacement field measurement process. The main defects are as follows: in the image data acquisition link in the early stage of the visual measurement task, due to the problems of the acquisition equipment or the limitation of the environment, the collected image data is blurred, ghosted or ringing, which is different from the real situation. In addition, due to the poor performance of the detection algorithm, the less feature information of the input image data and the low clarity of the input image data, the final fitting result of the visual detection algorithm may deviate from the real situation. How to improve the input image quality and enhance the input target feature information in the visual measurement task to strengthen the robustness of the algorithm is the key to realize efficient measurement, which has great significance for the development of engineering projects.

[0003] The existing visual-based measurement task rarely has a step of enhancing the input image data, which may result in excellent detection performance of these algorithms in a smooth scene, but when the scene changes, the detection accuracy may have a large fluctuation. How to realize the synchronous deblurring of the collected image and enhance the feature information of the image data in a complex scene to reduce the measurement error and improve the displacement detection performance is the primary research direction at present. SUMMARY

[0004] The present application provides an image deblurring network model and a visual measurement method for vibration displacement of a rotating body, which is used for deblurring the image of the rotating body and further realizes the displacement measurement of the target object in the image of the rotating body.

[0005] The technical scheme of the present application is as follows:

[0006] According to an aspect of the present application, an image deblurring network model is provided, comprising a shallow feature extraction module, a deep feature extraction module, a feature fusion module, a multi-scale feature extraction module, a dense residual module, an RDM module, and an attention mechanism; a blurred image is taken as the input of the first shallow feature extraction module, the output of the blurred image after a first downsampling operation is taken as the input of the first deep feature extraction module, the output of the blurred image after a second downsampling operation is taken as the input of the second deep feature extraction module, the output of the first shallow feature extraction module is taken as the input of the first dense residual module to obtain a first structure fusion feature map; the second shallow feature extraction module is added between the first dense residual module and the first feature fusion module, the output of the second shallow feature extraction module and the output of the first deep feature extraction module are taken as the input of the first feature fusion module, the output of the first feature fusion module is taken as the input of the second dense residual module to obtain a second structure fusion feature map; the third shallow feature extraction module is added between the second dense residual module and the second feature fusion module, the output of the third shallow feature extraction module and the output of the second deep feature extraction module are taken as the input of the second feature fusion module, the output of the second feature fusion module is taken as the input of the third dense residual module to obtain a third structure fusion feature map; the first, second and third structure fusion feature maps are taken as the input of the first multi-scale feature extraction module and the second multi-scale feature extraction module, the attention mechanism is added between the third dense residual module and the fourth dense residual module, the output of the fourth dense residual module is taken as the input of the RDM module, the output of the RDM module and the output of the second multi-scale feature extraction module are taken as the input of the fifth dense residual module after being aggregated, the output of the first multi-scale feature extraction module and the output of the fifth dense residual module are aggregated to obtain the input of the fourth shallow feature extraction module, the sixth dense residual module is added between the fourth shallow feature extraction module and the fifth shallow feature extraction module, the output of the fifth shallow feature extraction module is added to the blurred image to obtain a deblurred image.

[0007] The feature fusion module specifically comprises: taking the output of the deep feature extraction module as a feature map A and the output of the shallow feature extraction module as a feature map B; the feature map A is subjected to two independent convolution operations to obtain a feature map A-1 and a feature map A-2; the feature map A-2 and the feature map B are subjected to an addition operation to obtain a feature map B-1; the feature map B-1 is subjected to a convolution operation to obtain a feature map B-2; the feature map B-2 is subjected to a Sigmoid activation function to obtain a feature map B-3; the feature map A-1 and the feature map B-3 are subjected to a multiplication operation to obtain a feature map B-4; the feature map B-4 and the feature map A are subjected to an addition operation to obtain a fusion feature map B-5, which is taken as the output of the feature fusion module.

[0008] The dense residual module is obtained by connecting eight single-layer residual blocks in series.

[0009] The first multi-scale feature extraction module is configured to perform convolution operation on the aggregated first structure fusion feature map, the aggregated second structure fusion feature map, and the aggregated third structure fusion feature map, to obtain an output feature map of the first multi-scale feature extraction module; and the second multi-scale feature extraction module is configured to perform convolution operation on the aggregated first structure fusion feature map, the aggregated second structure fusion feature map, and the aggregated third structure fusion feature map, to obtain an output feature map of the second multi-scale feature extraction module.

[0010] The attention mechanism is specifically a coordinate attention mechanism.

[0011] According to another aspect of the present application, a method for measuring the vibration displacement of a rotating body by visualizing the deblurring of images is provided, which comprises: obtaining a rotating body vibration image dataset; obtaining a blurred image dataset according to the rotating body vibration image dataset; dividing the rotating body vibration image dataset and the blurred image dataset into training datasets and validation datasets; training the image deblurring network model using the training datasets of the rotating body vibration image dataset and the blurred image dataset, and screening the optimal candidate weight; evaluating the performance of the optimal candidate weight using the validation datasets of the rotating body vibration image dataset and the blurred image dataset, quantifying the performance of the weight, and loading the optimal weight obtained according to the quantification result into the proposed deblurring network model; deblurring the images in the blurred image dataset using the optimal weight to obtain a deblurred image dataset; labeling the target positions in the rotating body vibration image dataset, the blurred image dataset, and the deblurred image dataset to obtain corresponding label files; training the target detection network using the label files and the corresponding image data to obtain a weight file; and detecting the rotating body vibration image dataset, the blurred image dataset, and the deblurred image dataset using the trained weight file, respectively, and extracting the vibration displacement signals of the rotating body.

[0012] According to another aspect of the present application, there is provided a rotating body vibration displacement vision measurement system combined with image deblurring, comprising: an acquisition module configured to acquire a rotating body vibration image dataset; a first obtaining module configured to acquire a blurred image dataset based on the rotating body vibration image dataset; a division module configured to divide the rotating body vibration image dataset and the blurred image dataset into training datasets and validation datasets; a screening module configured to train the image deblurring network model using the training datasets of the rotating body vibration image dataset and the blurred image dataset, and screen out optimal candidate weights; a loading module configured to perform performance evaluation on the optimal candidate weights using the validation datasets of the rotating body vibration image dataset and the blurred image dataset, quantify the performance of the weights, load the optimal weights obtained according to the quantification results into the proposed deblurring network model; a second obtaining module configured to deblur the images in the blurred image dataset using the optimal weights to obtain a deblurred image dataset; a third obtaining module configured to label target positions in the rotating body vibration image dataset, the blurred image dataset and the deblurred image dataset to obtain corresponding label files; a fourth obtaining module configured to train a target detection network using the label files and the corresponding image data to obtain a weight file; and an extraction module configured to detect the rotating body vibration image dataset, the blurred image dataset and the deblurred image dataset using the trained weight file respectively, and extract the vibration displacement signals of the rotating body.

[0013] According to another aspect of the present application, there is provided a processor configured to run a program, wherein the program performs any one of the rotating body vibration displacement vision measurement methods combined with image deblurring.

[0014] The beneficial effects of the present application are: the present application proposes to use a feature fusion module, which can fuse deep and shallow feature information of the structure image to obtain more abundant structure feature information in the image data; further, the present application proposes a multi-scale feature extraction module, which can fuse structure image feature information under different scales to obtain better image reconstruction effect; on the other hand, the RDM module is used instead of the up-sampling method to retain more structure feature detail information, the module uses dense residual connection to fully utilize the feature map detail level information in different stages, in order to avoid the loss of feature information after image reconstruction of the feature map, the module can directly access the original input feature map at each layer, thereby realizing implicit deep supervision; based on the above, the deblurring network model proposed by the present application is used in the data collection stage of visual vibration measurement, which can improve the image data clarity and feature enhancement of the image data, effectively reducing the performance requirements such as frame rate and resolution of the camera required for image data collection; further, by comparing the displacement signal graph generated by the Ground-Truth image data, the displacement signal graph generated by the image data processed by the present application and the displacement signal graph generated by the blurred image data, the practical engineering application value of the present application is proved. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 It is the overall architecture diagram of the deblurring network of the present application.

[0016] Figure 2 It is a shallow feature extraction module structure diagram.

[0017] Figure 3 It is a feature fusion module structure diagram.

[0018] Figure 4 It is a multi-scale feature extraction module structure diagram.

[0019] Figure 5 It is a dense residual module structure diagram.

[0020] Figure 6 It is a coordinate attention mechanism structure diagram.

[0021] Figure 7 It is a feature fusion module effect display diagram.

[0022] Figure 8 It is a flowchart of the measurement method of the present application.

[0023] Figure 9 It is a time domain analysis comparison diagram of X-axis displacement signal and Y-axis displacement signal.

[0024] Figure 10 It is a time domain analysis comparison diagram of X-axis displacement signal and Y-axis displacement signal of blurred image and deblurred image. Detailed Implementation

[0025] The invention will be further described below with reference to the accompanying drawings and embodiments, but the scope of the invention is not limited to the description.

[0026] Example 1: As Figures 1-6 As shown, an image deblurring network model includes a shallow feature extraction module, a deep feature extraction module, a feature fusion module, a multi-scale feature extraction module, a dense residual module, an RDM (ResidualDense) module, and an attention mechanism. The model uses a blurred image as input to the first shallow feature extraction module, the output of the blurred image after a first downsampling operation as input to the first deep feature extraction module, and the output of the blurred image after a second downsampling operation as input to the second deep feature extraction module. The output of the first shallow feature extraction module is processed by the first dense residual module to obtain a first structure fusion feature map. A second shallow feature extraction module is added between the first dense residual module and the first feature fusion module. The outputs of the second shallow feature extraction module and the first deep feature extraction module are used as input to the first feature fusion module. The output of the first feature fusion module is processed by the second dense residual module to obtain a second structure fusion feature map. A third shallow feature extraction module is added between the second dense residual module and the second feature fusion module. The outputs of the third shallow feature extraction module and the second deep feature extraction module are used as input to the second deep feature fusion module. The first, second, and third structure fusion feature maps are input to the first and second multi-scale feature extraction modules. An attention mechanism is added between the third and fourth dense residual modules. The output of the fourth dense residual module is used as input to the RDM module. The outputs of the RDM module and the second multi-scale feature extraction module are aggregated and used as input to the fifth dense residual module. The outputs of the first multi-scale feature extraction module and the fifth dense residual module are aggregated and used as input to the fourth shallow feature extraction module. A sixth dense residual module is added between the fourth and fifth shallow feature extraction modules. The output of the fifth shallow feature extraction module is added to the blurred image to obtain the deblurred image.

[0027] Furthermore, the shallow feature extraction module (such as...) can be configured. Figure 2 As shown), it includes: using a single convolutional layer and activation function to construct a shallow feature extraction module to extract shallow feature information of the structure image; taking the collected blurred image data as input 1, first passing it through a convolution operation, and then through a ReLU activation function to obtain the features. Figure 1 .

[0028] Further, the deep feature extraction module can be configured to use Inception v3, and the blurred image data is input 1, which is subjected to 2 times and 4 times down-sampling processing to obtain input 1-2 and input 1-4, respectively. After the deep feature extraction module, the features Figure 2-1 and features Figure 2-2 are obtained, respectively.

[0029] Further, the feature fusion module (as shown in Figure 3 ) can be configured as follows: the output of the deep feature extraction module is taken as a feature map A, and the output of the shallow feature extraction module is taken as a feature map B; the feature map A is subjected to two independent convolution operations to obtain feature map A-1 and feature map A-2, respectively; the feature map A-2 and the feature map B are subjected to an addition operation to obtain feature map B-1; the feature map B-1 is subjected to a convolution operation to obtain feature map B-2; the feature map B-2 is subjected to a Sigmoid activation function to obtain feature map B-3; the feature map A-1 and the feature map B-3 are subjected to a multiplication operation to obtain feature map B-4; and the feature map B-4 and the feature map A are subjected to an addition operation to obtain a fusion feature map B-5, which is taken as the output of the feature fusion module.

[0030] Further, the first multi-scale feature extraction module can be configured to perform convolution operation on the aggregated feature maps of the first structure fusion feature map, the feature map subjected to the first up-sampling operation of the second structure fusion feature map, and the feature map subjected to the second up-sampling operation of the third structure fusion feature map, to obtain the output feature map of the first multi-scale feature extraction module; and the second multi-scale feature extraction module can be configured to perform convolution operation on the aggregated feature maps of the feature map subjected to the third down-sampling operation of the first structure fusion feature map, the second structure fusion feature map, and the feature map subjected to the third up-sampling operation of the third structure fusion feature map, to obtain the output feature map of the second multi-scale feature extraction module. As shown in Figure 4 , after the reconstruction operation of the structure fusion feature maps F1, F2 and F3, F 1-1 , F 2-1 and F 3-1 are obtained, respectively; F 1-1 , F 2-1 and F 3-1 are subjected to aggregation operation to obtain F4; and F4 is subjected to convolution operation to obtain F5. Specifically, for the first multi-scale feature extraction module, F2 and F3 are subjected to up-sampling operation to achieve the size consistent with F1; and for the second multi-scale feature extraction module, F1 and F3 are subjected to down-sampling operation and up-sampling operation, respectively, to achieve the size consistent with F2.

[0031] Further, the dense residual module can be configured to be obtained by connecting 8 single-layer residual blocks in series (as shown in Figure 5The output of the previous layer is the input of the next layer, and the input C is obtained after one convolution operation. The output C-1 is obtained after the ReLU activation function. The output C-2 is obtained after one convolution operation. The output C-3 is obtained after the addition operation of the output C-3 and the input C. The output D is obtained after one convolution operation. The output D-1 is obtained after the ReLU activation function. The output D-2 is obtained after one convolution operation. The output D-3 is obtained after the addition operation of the output D-3 and the output D. The output E is obtained after one convolution operation. The output E-1 is obtained after the ReLU activation function. The output E-2 is obtained after one convolution operation. The output E-3 is obtained after the addition operation of the output E-3 and the output E. The output F is obtained after one convolution operation. The output F-1 is obtained after the ReLU activation function. The output F-2 is obtained after one convolution operation. The output F-3 is obtained after the addition operation of the output F-3 and the output F. The output G is obtained after one convolution operation. The output G-1 is obtained after the ReLU activation function. The output G-2 is obtained after one convolution operation. The output G-3 is obtained after the addition operation of the output G-3 and the output G. The output H is obtained after one convolution operation. The output H-1 is obtained after the ReLU activation function. The output H-2 is obtained after one convolution operation. The output H-3 is obtained after the addition operation of the output H-3 and the output H. The output I is obtained after one convolution operation. The output I-1 is obtained after the ReLU activation function. The output I-2 is obtained after one convolution operation. The output I-3 is obtained after the addition operation of the output I-3 and the output I. The output J is obtained after one convolution operation. The output J-1 is obtained after the ReLU activation function. The output J-2 is obtained after one convolution operation. The output J-3 is obtained after the addition operation of the output J-3 and the output J. The output K is obtained after one convolution operation.

[0032] Further, the attention mechanism can be set as a coordinate attention mechanism (for example, as shown in FIG. 6B). Figure 6As shown in the figure), including: the attention mechanism can be respectively along the X axis and Y axis spatial direction of the input feature image feature aggregation of feature image; The specific implementation steps are as follows: the feature image after the feature fusion module and the dense residual module is taken as input P-A, and the image is averaged and pooled along the X axis direction and the Y axis direction respectively, and P1 and P2 are obtained respectively; We aggregate P1 and P2, input the aggregated data to the convolution operation to obtain P3; After normalization operation, P3 is divided into P4 and P5; P4 and P5 are respectively obtained after one convolution and Sigmoid activation function P4-1 and P5-1; P4-1, P5-1 and input P-A are multiplied to obtain output P-B.

[0033] It can be known from the above technical solution that, by adopting different down-sampling operations to participate in the deep feature extraction module, the deep feature information of the structural image under different scales is extracted, and the problem that the shallow feature information cannot be reconstructed in detail is solved. Further, the feature fusion module is arranged in two layers, which can fully utilize the deep and shallow feature information in the architecture to achieve better structural image reconstruction effect, such as Figure 7 As shown in the figure), including: the attention mechanism can be respectively along the X axis and Y axis spatial direction of the input feature image feature aggregation of feature image; The specific implementation steps are as follows: the feature image after the feature fusion module and the dense residual module is taken as input P-A, and the image is averaged and pooled along the X axis direction and the Y axis direction respectively, and P1 and P2 are obtained respectively; We aggregate P1 and P2, input the aggregated data to the convolution operation to obtain P3; After normalization operation, P3 is divided into P4 and P5; P4 and P5 are respectively obtained after one convolution and Sigmoid activation function P4-1 and P5-1; P4-1, P5-1 and input P-A are multiplied to obtain output P-B.

[0034] In order to further verify the advantages of the module combination of the present application, the image deblurring network model MSDNet proposed by the present application is experimented under the rotor (3500r / min) data set, and the experimental results are shown in Table 1. In the table, the symbol indicates that the module is used, and the symbol indicates that the corresponding module is a substitute module (using a single convolution layer instead of a deep feature extraction module, using up-sampling of corresponding parameters instead of RDM, and using feature addition instead of a feature fusion module):

[0035] Table 1

[0036]

[0037] From the data in Table 1, we can see that using RDM instead of upsampling can increase the image reconstruction accuracy by 0.82dB, and using the deep feature extraction module to extract features from the image can increase the final image reconstruction accuracy by 0.17dB. At the same time, using the feature fusion module to fuse deep and shallow features and adding a coordinate attention mechanism at the bottom of the network can also increase the image reconstruction accuracy by 0.01dB and 0.02dB respectively, proving that the modules used in this paper can better improve the structural image reconstruction accuracy.

[0038] Further, the present application selects multiple-input multiple-output U-Net (MIMO-UNet), multiple-input multiple-output U-Net+ (MIMO-UNet+), multi-stage progressive image restoration network (MPRNet), deep residual Fourier transform for single image deblurring (DeepRFT), and method for removing blind motion blur based on generative adversarial network method DeblurGANv2 for quantitative comparison between deblurring algorithms. We take the peak signal-to-noise ratio and structural similarity in the RGB channel when the model is tested on the test validation set to evaluate the performance of the network reconstructed image. The obtained weight file is tested on the rotor (3500r / min) dataset. From Table 2, it can be concluded that the MSDNET proposed in the present application performs well on most structural vibration image datasets.

[0039] Table 2

[0040]

[0041] Example 2: as Figure 1-10As shown, a kind of vibration displacement visual measurement method of rotating body combined with image deblurring includes: obtaining rotating body vibration image dataset;According to rotating body vibration image dataset, obtain blurred image dataset;Rotating body vibration image dataset, blurred image dataset are all divided into training dataset and verification dataset;Before formal training, the corresponding parameter in configuration file is modified, and appropriate loss function is selected;Rotating body vibration image dataset, the training dataset of blurred image dataset is used to train the above-mentioned image deblurring network model, and the optimal candidate weight is screened out;The optimal candidate weight is evaluated using the verification dataset of rotating body vibration image dataset, blurred image dataset, to quantify the performance of weight, and the optimal weight obtained according to the quantification result is loaded into the proposed deblurring network model;The optimal weight is used to deblur the image in blurred image dataset, and deblurred image dataset is obtained;The target position in rotating body vibration image dataset, blurred image dataset and deblurred image dataset is labeled, and the corresponding label file is obtained;The target detection network is trained using the label file and the corresponding image data, and the weight file is obtained;The trained weight file is used to detect rotating body vibration image dataset, blurred image dataset and deblurred image dataset respectively, and the vibration displacement signal of rotating body is extracted.

[0042] The loss function used in the deblurring stage is L1 loss function, which is defined as follows:

[0043]

[0044] Further, for the measurement method, the following optional implementation process is given:

[0045] Step 1, select the rotor as the target object, screw into the rotor on one side to artificially change the center of gravity, produce the phenomenon of unbalanced rotation of rotating body, and use high-speed camera to collect rotor vibration image data, set the frame rate to 2000 frames per second, the image size is 512x512, collect 1000 frames of Ground-Truth image data, i.e. rotating body vibration image dataset, in 0.5 seconds of unstable rotation of rotor, the high-speed rotating rotor image dataset is collected on the rotor vibration simulation test bed, and LED light compensation lamp is used for light compensation of the collected object. Then, the collected image data is subjected to Gaussian blur with σ=7 Gaussian kernel to simulate the problem of unclear collected image due to virtual focus or environmental reasons during collection, and 1000 frames of blurred image data are obtained after processing.

[0046] Step 2, divide the Ground-Truth image data and the blurred image data into training dataset and verification dataset according to the ratio of 7:3 respectively;

[0047] Step 3, use the shallow feature extraction module, deep feature extraction module, feature fusion module, multi-scale feature extraction module, dense residual module, RDM module, attention mechanism to construct the image deblurring network model;

[0048] Step 4, before formal training, modify the corresponding parameters in the configuration file, and select the appropriate loss function

[0049] Step 5, call the training data set and configuration file to start training the proposed deep learning network model, and select the optimal candidate weight after training;

[0050] Step 6, use the verification data set to evaluate the performance of the optimal candidate weight to quantify the performance of the weight, and load the optimal weight obtained from the quantization result into the proposed deblurring network model;

[0051] Step 7, use this weight to deblur the blurred image to obtain 1000 frames of deblurred image data; it should be noted that the deblurring network model can also be used for other images to deblur;

[0052] Step 8, take 500 Ground-Truth image data, blurred image data and deblurred image data respectively, and divide them into training set 1 and validation set 1, wherein the training set is 400 frames of image data and the validation set is 100 frames of image data. Use the labelimg labeling software to mark the boundary box of the rotor position in the training set 1 and the validation set 1 image, and generate a txt file, one image corresponds to one txt file, and the txt file contains the position information of the marked box in the image;

[0053] Step 9, input the image data and the corresponding generated txt file into the YOLOv5x target detection network for training to obtain the corresponding pt format weight file, and use the pt format weight file trained respectively to detect the Ground-Truth image data, blurred image data and deblurred image data;

[0054] Step 10, after detection, each frame of image will get the corresponding prediction box on the image, as well as a txt file containing the center point of the prediction box. Summarize the center point coordinates corresponding to each frame of image, and draw a curve graph using the data obtained by summarizing, which is the vibration displacement signal of the high-speed vibrating rotor.

[0055] Through the comparison of Figure 9 , 10 , it can be seen that the rotor vibration displacement signal regressed from the deblurred image data is more accurate than the rotor vibration displacement signal data regressed from the blurred image data. Among them, Figure 10 is from Figure 9The two curves intersect.

[0056] The accuracy of the displacement signal of the high-speed rotating rotor is closely related to the detection edge accuracy, and the detection edge accuracy and the extraction of the refined edge features of the image are related to the definition of the collected image data. The boundary of the image without the deblurring network of the present application is relatively blurred, and the definition is low, which will affect the subsequent accuracy of the displacement signal. The present application solves the above problems well. Specifically, in the visual vibration measurement pre-image data acquisition step, when collecting high-speed vibration body images, improper camera parameter setting or poor collection site environment causes the collected image data to be blurred, ghosted or ringing, resulting in a certain deviation between the collected image data and the true situation. And the boundary of the unprocessed image is relatively blurred, which further leads to the subsequent inability to obtain refined edge features and the poor accuracy of the generated displacement vibration signal. The network model proposed by the present application starts from the perspective of improving the definition of the input image in the visual vibration measurement task, aggregates the feature information at different scales in the encoding-decoding architecture to obtain more detailed feature information of the structure, and further focuses on the reconstruction of the high-frequency information of the frame. The vibration image data after restoring the high-frequency components is used as the input, the feature extraction algorithm is used to automatically count or analyze the vibration region feature information, the vibration video image features are effectively extracted, finally, the selected image features are used to track the target structure or marker, the position of the target in each frame of image is determined, the target position in each frame of image is summarized, the target position of the structure is accurately tracked, and the accuracy of the displacement signal is improved.

[0057] In the embodiment 3, the image deblurring combined with the visual measurement system of the vibration displacement of the rotating body comprises: an acquisition module, configured to acquire a rotating body vibration image dataset; a first obtaining module, configured to acquire a blurred image dataset according to the rotating body vibration image dataset; a division module, configured to divide the rotating body vibration image dataset and the blurred image dataset into training datasets and verification datasets; a screening module, configured to train the image deblurring network model by using the training datasets of the rotating body vibration image dataset and the blurred image dataset, and screen out the optimal candidate weight; a loading module, configured to perform performance evaluation on the optimal candidate weight by using the verification datasets of the rotating body vibration image dataset and the blurred image dataset, to quantify the performance of the weight, load the optimal weight obtained according to the quantification result into the proposed deblurring network model; a second obtaining module, configured to deblur the images in the blurred image dataset by using the optimal weight, and obtain a deblurred image dataset; a third obtaining module, configured to label target positions in the rotating body vibration image dataset, the blurred image dataset and the deblurred image dataset, and obtain corresponding label files; a fourth obtaining module, configured to train a target detection network by using the label files and the corresponding image data, and obtain a weight file; and an extraction module, configured to detect the rotating body vibration image dataset, the blurred image dataset and the deblurred image dataset by using the trained weight file respectively, and extract the vibration displacement signals of the rotating body.

[0058] In the embodiment 4, a processor is used to run a program, wherein the program performs the image deblurring combined with the visual measurement method of the vibration displacement of the rotating body.

[0059] The above embodiment numbers of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0060] In the above embodiments of the application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0061] The specific embodiments of the application are described in detail above in combination with the drawings, but the application is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the application.

Claims

1. A method for constructing an image deblurring network model, characterized in that, The image deblurring network model comprises a shallow feature extraction module, a deep feature extraction module, a feature fusion module, a multi-scale feature extraction module, a dense residual module, an RDM module and an attention mechanism; a blurred image is taken as an input of the first shallow feature extraction module, an output of the blurred image after a first downsampling operation is taken as an input of the first deep feature extraction module, an output of the blurred image after a second downsampling operation is taken as an input of the second deep feature extraction module, an output of the first shallow feature extraction module is taken as an input of the first dense residual module, an output of the first dense residual module is taken as an input of the first feature fusion module, an output of the first feature fusion module is taken as an input of the second dense residual module, an output of the second dense residual module is taken as an input of the second feature fusion module, an output of the second feature fusion module is taken as an input of the third dense residual module, an output of the third dense residual module is taken as an input of the RDM module, an output of the RDM module and an output of the second multi-scale feature extraction module are taken as inputs of the fifth dense residual module after being aggregated, an output of the first multi-scale feature extraction module and an output of the fifth dense residual module are aggregated and taken as an input of the fourth shallow feature extraction module, the fourth shallow feature extraction module and the fifth shallow feature extraction module are connected through the sixth dense residual module, an output of the fifth shallow feature extraction module is added to the blurred image to obtain a deblurred image.

2. The method of claim 1, wherein, The feature fusion module specifically comprises: an output of the deep feature extraction module is taken as a feature map A, and an output of the shallow feature extraction module is taken as a feature map B; the feature map A is subjected to two independent convolution operations to obtain a feature map A-1 and a feature map A-2 respectively; the feature map A-2 and the feature map B are subjected to an addition operation to obtain a feature map B-1; the feature map B-1 is subjected to a convolution operation to obtain a feature map B-2; the feature map B-2 is subjected to a Sigmoid activation function to obtain a feature map B-3; the feature map A-1 and the feature map B-3 are subjected to a multiplication operation to obtain a feature map B-4; the feature map B-4 and the feature map A are subjected to an addition operation to obtain a fusion feature map B-5, which is taken as an output of the feature fusion module.

3. The method of claim 1, wherein the image deblurring network model is constructed by, The dense residual module is obtained by connecting eight single-layer residual blocks in series.

4. The method according to claim 1, further comprising: The first multi-scale feature extraction module is configured to perform convolution operation on the aggregated first structure fusion feature map, the second structure fusion feature map after the first up-sampling operation, and the third structure fusion feature map after the second up-sampling operation, to obtain an output feature map of the first multi-scale feature extraction module. The second multi-scale feature extraction module is configured to perform convolution operation on the aggregated first structure fusion feature map after the third down-sampling operation, the second structure fusion feature map, and the third structure fusion feature map after the third up-sampling operation, to obtain an output feature map of the second multi-scale feature extraction module.

5. The method of claim 1, wherein the image deblurring network model is constructed by, The attention mechanism is specifically a coordinate attention mechanism.

6. A method for visual measurement of rotational body vibration displacement by deblurring of combined images, characterized in that The method comprises the following steps: obtaining a rotating body vibration image dataset; obtaining a blurred image dataset according to the rotating body vibration image dataset; dividing the rotating body vibration image dataset and the blurred image dataset into training datasets and verification datasets; training the image deblurring network model constructed by the construction method of claim 1 using the training datasets of the rotating body vibration image dataset and the blurred image dataset to screen out optimal candidate weights; performing performance evaluation on the optimal candidate weights using the verification datasets of the rotating body vibration image dataset and the blurred image dataset to quantify the performance of the weights, and loading the optimal weights obtained according to the quantification results into the proposed deblurring network model; performing deblurring on images in the blurred image dataset using the optimal weights to obtain a deblurred image dataset; annotating target positions in the rotating body vibration image dataset, the blurred image dataset, and the deblurred image dataset to obtain corresponding annotation files; training a target detection network using the annotation files and corresponding image data to obtain a weight file; detecting the rotating body vibration image dataset, the blurred image dataset, and the deblurred image dataset using the trained weight file respectively, and extracting vibration displacement signals of the rotating body.

7. A rotating body vibration displacement vision measurement system combined with image deblurring, characterized by, The method comprises the following steps: an obtaining module is configured to obtain a rotating body vibration image dataset; a first obtaining module is configured to obtain a blurred image dataset according to the rotating body vibration image dataset; a dividing module is configured to divide the rotating body vibration image dataset and the blurred image dataset into training datasets and verification datasets; a screening module is configured to train the image deblurring network model constructed by the construction method of claim 1 using the training datasets of the rotating body vibration image dataset and the blurred image dataset to screen out optimal candidate weights; a loading module is configured to perform performance evaluation on the optimal candidate weights using the verification datasets of the rotating body vibration image dataset and the blurred image dataset to quantify the performance of the weights, and load the optimal weights obtained according to the quantification results into the proposed deblurring network model; a second obtaining module is configured to perform deblurring on images in the blurred image dataset using the optimal weights to obtain a deblurred image dataset; a third obtaining module is configured to annotate target positions in the rotating body vibration image dataset, the blurred image dataset, and the deblurred image dataset to obtain corresponding annotation files; a fourth obtaining module configured to train the target detection network by using the annotation file and the corresponding image data to obtain a weight file; an extracting module configured to detect the rotating body vibration image dataset, the blurred image dataset and the deblurred image dataset by using the trained weight file respectively, and extract the vibration displacement signal of the rotating body.

8. A processor, comprising: The processor is configured to run a program, and the program performs the method for visual measurement of vibration displacement of a rotating body combined with image deblurring when running.

Images