A Monitoring Method for Helicopter Blade Vibration Quantity Based on Deep Learning Dichotomy

Abstract

This application relates to a method for monitoring the vibration quantity of helicopter blades based on deep learning dichotomy, which includes the following steps: establishing the mapping relationship between two-dimensional pixel coordinates and three-dimensional world coordinates; collecting real-time blade vibration images; constructing an improved YOLOv8n network, and using deep learning dichotomy to complete the circular detection of blade marking points, specifically including using the improved YOLOv8n network to locate the blade marking points and using the ellipse segmentation network to segment the blade marking points to obtain the center pixel coordinates of the blade marking points; collecting the initial positions of the blade marking points; and real-time calculating the vibration quantity of the blades. The present invention can solve the problems of poor adaptability in complex environments, low positioning accuracy, and insufficient real-time processing performance of traditional visual monitoring methods, realizes highly robust real-time monitoring of millimeter-level vibration quantity, and provides a reliable technical means for the fatigue assessment of the entire life cycle of helicopter blades.

Description

Technical Field

[0001] The present application relates to the technical field of computer vision real-time monitoring, and specifically to a method for monitoring helicopter blade vibration based on deep learning dichotomy. Background Art

[0002] As a core power component, the dynamic characteristics of helicopter blades directly affect flight safety. Traditional blade vibration monitoring methods are mainly divided into contact and non-contact methods: contact measurement usually uses strain gauges or accelerometers directly fixed on the blade surface. Although local strain information can be obtained, there are problems such as additional mass changing the structural dynamic characteristics and complex wiring affecting the operation of rotating parts; in non-contact measurement, optical measurement based on machine vision has gradually become a research hotspot due to its advantages such as full-domain measurement and no additional mass.

[0003] Among the existing visual measurement methods, the commonly used one is the marker tracking technology based on traditional image processing. This method arranges high-contrast circular markers on the surface of the blade, uses Canny edge detection combined with the least squares ellipse fitting algorithm to extract the coordinates of the center of the circle, and then calculates the three-dimensional vibration through coordinate mapping. However, this method has significant defects: 1. It relies on ideal lighting conditions. Under complex ambient light or shadow interference, edge detection is easily affected by noise, resulting in increased ellipse fitting errors; 2. When the markers are blurred or partially blocked due to high-speed rotation, it is difficult for traditional algorithms to accurately segment the marker area, and the accuracy of center positioning drops sharply, with the error often exceeding 2 pixels; 3. The processing flow lacks adaptive capabilities, and the threshold parameters need to be adjusted repeatedly by humans. The single-frame processing time is >30ms, which cannot meet the real-time requirements when the vibration frequency exceeds 5Hz.

[0004] Although some studies in recent years have attempted to introduce general detection networks such as Faster R-CNN network and YOLOv5 network into vibration monitoring, there are still the following shortcomings: 1. The network structure is redundant. The detection head is designed for multi-scale targets, while the blade marker point size is single, resulting in a waste of computing resources; 2. The traditional convolution module has a large number of parameters, making it difficult to achieve real-time inference in embedded devices; 3. The end-to-end method of directly regressing the center coordinates lacks sub-pixel positioning capabilities, and the measurement error is generally higher than 1.5mm, which cannot meet the needs of high-precision fatigue testing. Summary of the invention

[0005] The purpose of the present invention is to provide a helicopter blade vibration monitoring method based on deep learning dichotomy, which can solve the problems of poor adaptability of traditional visual monitoring methods in complex environments, low positioning accuracy and insufficient real-time processing performance, and realize highly robust real-time monitoring of millimeter-level vibration, providing a reliable technical means for fatigue assessment of helicopter blades throughout their life cycle.

[0006] The technical solution adopted in this application is as follows: A method for monitoring the vibration amount of helicopter blades based on deep learning dichotomy, including the following steps:

[0007] Step S1: Establish a mapping relationship from two-dimensional pixel coordinates to three-dimensional world coordinates;

[0008] Step S2: Collect real-time blade vibration images;

[0009] Step S3: Construct an improved YOLOv8n network, replace all traditional convolution modules in the Bottleneck modules of the YOLOv8n network with lightweight convolution modules, and delete the large-object detection head of the YOLOv8n network; use deep learning dichotomy to complete the circular detection of blade marking points, specifically including using the improved YOLOv8n network to locate the blade marking points and using the ellipse segmentation network to segment the blade marking points to obtain the center pixel coordinates of the blade marking points;

[0010] Step S4: Collect the initial positions of the blade marking points;

[0011] Step S5: Calculate the blade vibration amount in real time.

[0012] Furthermore, the mapping relationship from two-dimensional pixel coordinates to three-dimensional world coordinates in step S1 is:

[0013] ;

[0014] where, ( u , v ) are the two-dimensional pixel coordinates of the marking points on the blade, z c is the distance from the marking point to the origin along the optical axis in the camera coordinate system, K is the internal parameter matrix of the camera, R is the rotation matrix from the camera two-dimensional pixel coordinate system to the three-dimensional world coordinate system of the vibration plane, t is the translation matrix from the camera two-dimensional pixel coordinate system to the three-dimensional world coordinate system of the vibration plane, ( x w , y w , z w ) are the three-dimensional world coordinates of the marking point in the three-dimensional world coordinate system of the vibration plane.

[0015] Furthermore, the lightweight convolution module includes a channel splitting module, a two-dimensional convolution module, and a channel splicing module; the input feature map first passes through the channel splitting module and is split into feature Figure 1 and feature Figure 2 in proportion, and feature Figure 1 is subjected to feature extraction through the two-dimensional convolution module and then combined with featureFigure 2 Channel splicing is performed to obtain an output feature map.

[0016] Further, the specific steps of step S3 are as follows:

[0017] Step S301: Use the LabelImg annotation tool to annotate the marked points on the real-time blade vibration image collected in step S2 to obtain the corresponding label file, and form a marked point detection data set with the blade vibration image and the corresponding label file;

[0018] Step S302: Train the improved YOLOv8n marked point detection network, and input the marked point detection data set into the trained improved YOLOv8n marked point detection network for recognition, and output the position information of the marked points;

[0019] Step S303: Crop the blade marked point image from the blade vibration image, and perform binary segmentation on the cropped marked point image to obtain a marked point binary image with the same size as the cropped blade marked point image, and form a marked point segmentation data set with the blade marked point image and the corresponding marked point binary image;

[0020] Step S304: Construct an ellipse segmentation network, which includes three parts: an encoding network D1, a decoding network D2, and an output network D3; the encoding network D1 includes an input layer and an encoding layer, and the encoding layer includes three double-layer convolutional layers and three pooling layers arranged alternately; the decoding network D2 includes three upsampling layers and three double-layer convolutional layers arranged alternately, and each upsampling layer is connected in a skip connection with the double-layer convolutional layer in the encoding network D1; the output network D3 includes a pointwise convolutional layer; wherein, the double-layer convolutional layer includes an input layer, two convolutional layers arranged alternately, two batch normalization layers, two activation function layers, and an output layer;

[0021] Step S305: Train the ellipse segmentation network. During the training process, use binary cross-entropy loss as the loss function to calculate the marked point segmentation error, and backpropagate the error to update the network parameters to achieve network convergence. Continuously train the network until the error between the predicted marked point center coordinates and the true center coordinates is within 0.5 pixels, and save a set of model parameters with the smallest loss value or the smallest average center error during the training process according to the calculated segmentation error;

[0022] Step S306: Load the ellipse segmentation network model and set the hyperparameters of the network. Crop all circular marker point images from the blade vibration image according to the position information of the marker points obtained in Step S302, and scale them to the same size as the cropped blade marker point image. Then input them into the ellipse segmentation network for processing. The ellipse segmentation network outputs the binary image of each marker point and performs ellipse fitting on the binary image to obtain the center coordinates of all marker points. According to the scaling relationship during the cropping of the marker points, transform the center coordinates to the original image to obtain the two-dimensional pixel coordinates of the centers of each circular marker point in the original image.

[0023] Furthermore, the calculation formula of the loss function is as follows:

[0024] ;

[0025] where YP represents the predicted value output by the ellipse segmentation network; YT represents the true value corresponding to the input image; Loss ( YT, YP ) represents the loss function, that is, the segmentation error between the network predicted value and the true value. yp ij represents the pixel value at the i -th row j and yt ij -th column i of the predicted image; j represents the pixel value at the

[0026] -th row XS k and YS k -th column k = 1, 2, …, n , n represents the number of marker points on the blade, XS k represents the initial three-dimensional abscissa of the k -th marker point, YS k represents the initial three-dimensional ordinate of the k -th marker point.

[0027] Further, the specific steps of step S5 are as follows: Use an industrial camera to capture the vibration images of the blade in real time, and use the center positioning method based on deep learning dichotomy to obtain the two-dimensional pixel coordinates of the centers of all marked points in the vibration image. Then, according to the mapping relationship from the two-dimensional pixel coordinates to the three-dimensional world coordinates, convert all the two-dimensional pixel coordinates of the centers into three-dimensional world coordinates ( XE k , YE k , 0), where k = 1, 2, …, n , k represents the serial number of the marked point, n represents the number of marked points on the blade, XE k represents the k th three-dimensional abscissa of a marked point at a certain moment during the vibration, YE k represents the k th three-dimensional ordinate of a marked point at a certain moment during the vibration; Calculate the blade vibration amount = | YE k - YS k |, and complete the calculation of the vibration amount at each position of the blade.

[0028] The beneficial effects of this application are as follows: The present invention adopts a center positioning method based on deep learning dichotomy, which can locate the center of the marked point in a complex lighting environment, and the positioning accuracy rate reaches 99%; The present invention adopts monocular vision technology. By obtaining the mapping relationship from the two-dimensional pixel coordinates on the camera image to the three-dimensional world coordinates on the blade vibration plane, while realizing non-contact measurement, the measurement error is controlled within 1 mm; When the blade vibration frequency is about 6 Hz, it can realize the real-time monitoring of the blade vibration amount; The present invention combines the object detection and object segmentation technologies based on deep learning with computer vision technology. When monitoring the vibration amount in real time, it has the advantages of non-contact, strong adaptability and high precision, and can be applied to the monitoring of the blade vibration amount in fatigue tests, which has important practical value for the design and research and development of helicopter blades. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0030] Figure 1 is the method flow chart of the embodiment of the present invention;

[0031] Figure 2 This is a schematic structural diagram of the real-time acquisition device for blade vibration images in the embodiments of the present invention;

[0032] Figure 3 This is a schematic structural diagram of the lightweight convolution module in the embodiments of the present invention;

[0033] Figure 4 This is a structural diagram of the ellipse segmentation network in the embodiments of the present invention;

[0034] Figure 5 This is a schematic diagram of the blade vibration amount in the embodiments of the present invention;

[0035] Figure 6 This is a segmentation result diagram of the marked points in the embodiments of the present invention. Among them, (a), (c), and (e) are marked images under different background lighting environments, and (b), (d), and (f) are segmentation result diagrams under different background lighting environments;

[0036] Figure 7 This is a schematic structural diagram of the vibration amount detection and verification device in the embodiments of the present invention.

[0037] Explanation of reference numerals: 101 - processing computer, 102 - industrial camera, 103 - blade, 104 - blade marked point. Detailed implementation manners

[0038] In order to be able to more clearly understand the above objects, features, and advantages of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners. Many specific details are set forth in the following description in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the present invention is not limited by the limitations of the specific embodiments disclosed below.

[0039] As Figure 1 shown, the embodiments of the present invention propose a method for monitoring the vibration amount of helicopter blades based on the deep learning dichotomy method, including the following steps:

[0040] Step S1: Establish a mapping relationship from two-dimensional pixel coordinates to three-dimensional world coordinates. In the embodiments of the present invention, a processing computer 101 and an industrial camera 102 are used to construct as Figure 2The real-time acquisition device for the blade vibration image shown, the CPU model of the processing computer 101 is Intel(R) Core (TM) i7-8750H, the GPU model is GTX 1070, the video memory is 8G, the memory is 24G, the operating system is WIN10, and the disk capacity is 256GB solid-state drive + 1T mechanical hard drive. The industrial camera 102 is a German IDS digital industrial camera, the lens is a Japanese RICOH lens, the focal length is 12mm, the circular marking points in the calibration board are 8 rows and 7 columns, the center distance is 35mm, the size of the collected image is fixed at 1280×1024, keeping the calibration board on the blade vibration plane and within the camera's field of view, calculating the mapping relationship from the two-dimensional pixel coordinates of the imaging plane to the three-dimensional world coordinates of the vibration plane, that is, the rotation matrix and the translation matrix. The specific method is as follows:

[0041] Coincide the calibration board on the vibration plane of the blade 103, and keep the calibration board within the shooting field of view of the industrial camera 102. Input parameters such as the calibrated center distance, number of rows of circles, and number of columns of circles in the processing computer 101, and use the camera to collect 10 calibration board images. Perform calibration processing on the 10 calibration board images obtained by shooting through the Zhang's calibration method to obtain the corresponding reprojection error, rotation matrix, and translation matrix. Select the image with the smallest calibration reprojection error, and take its corresponding rotation matrix and translation matrix as the rotation matrix and translation matrix from the camera two-dimensional pixel coordinate system to the vibration plane three-dimensional world coordinate system. The mapping relationship from two-dimensional pixel coordinates to three-dimensional world coordinates is;

[0042] ;

[0043] Among them, ( u , v ) is the two-dimensional pixel coordinate of the blade marking point 104, z c is the distance from the marking point to the origin along the optical axis direction in the camera coordinate system, K is the internal parameter matrix of the camera, R is the rotation matrix from the camera two-dimensional pixel coordinate system to the vibration plane three-dimensional world coordinate system, t is the translation matrix from the camera two-dimensional pixel coordinate system to the vibration plane three-dimensional world coordinate system, ( x w , y w , z w ) is the three-dimensional world coordinate of the marking point in the vibration plane three-dimensional world coordinate system. The Zhang's calibration method stipulates that the calibration board plane z w = 0, substitute z w = 0 into the formula to eliminate zc , the three-dimensional world coordinates of the marked points are obtained by solving ( x w , y w , 0), completing the mapping from two-dimensional pixel coordinates to three-dimensional world coordinates.

[0044] Step S2: Collect real-time blade vibration images. In the example of the present invention, a plastic sheet is cut into the shape of a helicopter blade as a simulated blade, and 5 circular marked points are pasted on the simulated blade, that is, the number of marked points n =5, the diameter of the marked point is 3.5 cm, and the color is silver-white. The trigger mode of the industrial camera 102 is set to internal trigger, the camera acquisition frame rate is set to 50 frames per second, and the camera imaging plane forms a 45-degree angle with the blade vibration plane. During the blade vibration process, the industrial camera 102 continuously captures vibration images.

[0045] Step S3: Construct an improved YOLOv8n network, replace the traditional convolutional modules in all Bottleneck modules in the YOLOv8n network with lightweight convolutional modules, and delete the large object detection head of the YOLOv8n network; use the deep learning dichotomy method to complete the circular detection of the blade marked points, specifically including using the improved YOLOv8n network to locate the blade marked points and using the ellipse segmentation network to segment the blade marked points to obtain the center pixel coordinates of the blade marked points.

[0046] The specific steps of step S3 are as follows:

[0047] Step S301: Use the LabelImg annotation tool to annotate the marked points on the real-time blade vibration images collected in step S2 to obtain the corresponding label files, and form a marked point detection data set from the blade vibration images and the corresponding label files. In the embodiment of the present invention, the blade vibration image acquisition device as Figure 2 shown is placed in different lighting environments to construct a blade vibration test simulation environment, the blade vibration images in different lighting environments are collected and the corresponding label files are obtained, the blade vibration images and the corresponding label files are placed in different folders in the same directory, forming a marked point detection data set, and at the same time divided into a training set and a test set according to a ratio of 4:1.

[0048] Step S302: Train the improved YOLOv8n marked point detection network, and input the marked point detection data set into the trained improved YOLOv8n marked point detection network for recognition, and output the position information of the marked points. The lightweight convolutional module in the improved YOLOv8n marked point detection network is as Figure 3As shown in the figure, b in the figure represents the number of images input to the network at one time, h represents the height of the image, w represents the width of the image, and c represents the number of channels of the image. The lightweight convolution module includes a channel splitting module, a two-dimensional convolution module, and a channel splicing module; the input feature map first passes through the channel splitting module and is split into features according to a ratio Figure 1 and feature Figure 2 , feature Figure 1 After feature extraction by the two-dimensional convolution module, it is concatenated with feature Figure 2 to obtain the output feature map through channel splicing. In the embodiment of the present invention, the input feature map is split into feature Figure 1 and feature Figure 2 in a ratio of 1:3.

[0049] Step S303: Crop the blade marker point image from the blade vibration image, resize the image to 120×120 pixel size, and perform binary segmentation on the cropped blade marker point image to obtain a marker point binary image with the same size as the cropped blade marker point image. Then, put the blade marker point image and the corresponding marker point binary image into different folders in the same directory to form a marker point segmentation dataset, and randomly divide it into a training set and a test set according to a ratio of 4:1.

[0050] Step S304: Construct an ellipse segmentation network. The structure of the ellipse segmentation network is as Figure 4 shown, including three parts: an encoding network D1, a decoding network D2, and an output network D3; the encoding network D1 includes an input layer and an encoding layer, and the encoding layer includes three double-layer convolutional layers and three pooling layers arranged alternately; the decoding network D2 includes three upsampling layers and three double-layer convolutional layers arranged alternately, and each upsampling layer is connected in a skip connection with the double-layer convolutional layer in the encoding network D1; the output network D3 includes a pointwise convolutional layer; among them, the double-layer convolutional layer includes an input layer, two convolutional layers arranged alternately, two batch normalization layers, two activation function layers, and an output layer. In the figure, in represents the number of channels of the input tensor, mid represents the number of channels during the tensor processing, and out represents the number of channels of the output tensor. In the embodiment of the present invention, the scaling factor of the upsampling layer is set to 2, and the bilinear interpolation algorithm is selected for the upsampling algorithm.

[0051] Step S305: Train the ellipse segmentation network. During the training process, set the number of training epochs to 150, the learning rate to 1E-4, and the training batch size to 20. Use binary cross-entropy loss as the loss function to calculate the marker point segmentation error, and perform backpropagation on the error to update the network parameters to achieve network convergence. Continuously train the network until the error between the predicted center coordinates of the marker points and the true center coordinates is within 0.5 pixels, and save a set of model parameters with the smallest loss value or the smallest average center error during the training process according to the calculated segmentation error.

[0052] The calculation formula of the loss function is as follows:

[0053] ;

[0054] where, YP represents the predicted value output by the ellipse segmentation network; YT represents the ground truth value corresponding to the input image; Loss ( YT, YP ) represents the loss function, that is, the segmentation error between the network predicted value and the ground truth value, yp ij represents the pixel value at the i -th row j and yt ij -th column of the predicted image; i represents the row j and

[0055] represents the pixel value at the

[0056] -th column of the ground truth image. Step S306: Load the ellipse segmentation network model and set the hyperparameters of the network. Then, crop all the circular marker point images from the blade vibration image according to the position information of the marker points obtained in step S302, and scale them to the same size as the cropped blade marker point image, that is, 120×120 pixel size. Subsequently, input them into the ellipse segmentation network for processing. The ellipse segmentation network outputs the binary image of each marker point and performs ellipse fitting on the binary image to obtain the center coordinates of all the marker points. According to the scaling relationship during the cropping of the marker points, transform the center coordinates to the original image to obtain the two-dimensional pixel coordinates of the centers of the circular marker points in the original image. XS k , YS k , 0) and save them, where, k = 1, 2, …, n , n represents the number of marker points on the blade, XS k represents the initial three-dimensional abscissa of the k -th marker point, YS k represents the initial three-dimensional ordinate of the k -th marker point.

[0057] Step S5: Calculate the blade vibration amount in real time. The specific steps are as follows: As Figure 5 shown, use the industrial camera 102 to capture the blade vibration images in real time, and use the center positioning method based on deep learning dichotomy to obtain the two-dimensional pixel coordinates of the centers of all marked points in the vibration images. According to the mapping relationship from two-dimensional pixel coordinates to three-dimensional world coordinates, convert all two-dimensional pixel coordinates of the centers into three-dimensional world coordinates ( XE k , YE k , 0), where k = 1, 2, …, n , k represents the serial number of the marked point, n represents the number of marked points on the blade, XE k represents the k th three-dimensional abscissa of the YE k th marked point at a certain moment during the vibration process, k represents the th three-dimensional ordinate of the YE k th marked point at a certain moment during the vibration process; the hollow circles in the figure represent the initial positions of the blade marked points, and the solid circles represent the positions of the blade marked points at a certain moment. Calculate the blade vibration amount YS k = |

[0058] -

[0059] | to complete the calculation of the vibration amount at each position of the blade.

[0060] Next, the performance indicators of the monitoring method described in the embodiments of the present invention will be illustrated through experiments. Figure 6 Place the simulated blade under the test images with different brightness and complex backgrounds as

[0061] shown for experiments. In the figure, (a), (c), and (e) are the marked point images with different brightness, and (b), (d), and (f) are the corresponding segmentation result images of the marked points. It can be seen from the experiments that the embodiments of the present invention can well segment the marked points and fit the centers of the marked points under different brightness and complex background conditions, and have strong robustness.

[0062] The training effect of the ellipse segmentation network in the examples of the present invention is shown in Table 1.

[0063] Table 1 Ellipse segmentation network training index data

[0064]

[0065] During the network training process, 5 groups of data are averaged out from the 0 - 150 training processes. The higher the pixel accuracy and the smaller the pixel error, the more accurate the marker point segmentation network is for the marker points, and the more accurate the center detection is. From the data in Table 1, on the validation set, the pixel error of the center is maintained within 0.5 pixels, indicating that the marker point segmentation model has a high accuracy.

[0066] Using a device with known center - distance marker points pasted as Figure 7 shown to verify the measurement accuracy of the embodiments of the present invention, the detection results as shown in Table 2 can be obtained.

[0067] Table 2 Verification of the Measurement Accuracy of the Vibration Quantity Monitoring Method

[0068]

[0069] From the data in Table 2, it can be seen that the embodiments of the present invention can achieve high - precision detection of the center of circular marker points, and the average error of the vibration quantity can be controlled within 1 mm, fully meeting the monitoring requirements of the blade vibration quantity.

[0070] 3. Center location time test.

[0071] Considering that when the helicopter blade is undergoing a fatigue test, the vibration frequency is about 6 Hz. To achieve real - time vibration monitoring, the time required for a series of operations such as the acquisition, processing, and result visualization of each vibration image needs to be controlled within 20 ms, and the processing frame rate needs to be controlled above 50 frames. In the embodiments of the present invention, the number of blade marker points is 5, and the experimental center location time data is shown in Table 3.

[0072] Table 3 Center Location Time

[0073]

[0074] Among them, when the GTX 1070 - type GPU calls the improved YOLOv8n marker point detection network and the ellipse segmentation network, it takes a total of 15 ms from the input image to output 5 center coordinates, while the RTX 4060Ti - type GPU takes a total of 6 ms, fully meeting the real - time monitoring requirements of the blade vibration quantity.

[0075] The above - mentioned are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A helicopter blade vibration monitoring method based on deep learning dichotomy, characterized in that: The steps include: Step S1: Establishing a mapping relationship from two-dimensional pixel coordinates to three-dimensional world coordinates; Step S2: collecting real-time blade vibration images; Step S3: construct an improved YOLOv8n network, use lightweight convolution modules to replace the traditional convolution modules in all Bottleneck modules in the YOLOv8n network, and delete the large target detection head of the YOLOv8n network; use deep learning dichotomy to complete the circular detection of the blade markers, specifically including using the improved YOLOv8n network to locate the blade markers and using the ellipse segmentation network to segment the blade markers to obtain the pixel coordinates of the center of the blade markers; the ellipse segmentation network includes an encoding network D1, a decoding network D2, and a decoding network D3. The encoding network D1 includes an input layer and an encoding layer, and the encoding layer includes three double-layer convolutional layers and three pooling layers that are alternately arranged; the decoding network D2 includes three upsampling layers and three double-layer convolutional layers that are alternately arranged, and each upsampling layer is jump-connected to the double-layer convolutional layer in the encoding network D1; the output network D3 includes a point-by-point convolutional layer; wherein the double-layer convolutional layer includes an input layer, two convolutional layers that are alternately arranged, two batch normalization layers, two activation function layers, and an output layer; Step S4: collecting the initial position of the blade marking point; Step S5: Calculate the blade vibration in real time.

2. A helicopter blade vibration monitoring method based on deep learning dichotomy according to claim 1, characterized in that: The mapping relationship from the two-dimensional pixel coordinates to the three-dimensional world coordinates in step S1 is: in,( u , v ) is the two-dimensional pixel coordinate of the marking point on the blade, z c is the distance from the marking point to the origin along the optical axis in the camera coordinate system, K is the camera’s intrinsic parameter matrix, R is the rotation matrix from the camera's 2D pixel coordinate system to the vibration plane's 3D world coordinate system, t is the translation matrix from the camera's 2D pixel coordinate system to the vibration plane's 3D world coordinate system, ( x w , y w , z w ) is the three-dimensional world coordinate of the marking point in the three-dimensional world coordinate system of the vibration plane.

3. A helicopter blade vibration monitoring method based on deep learning dichotomy according to claim 2, characterized in that: The lightweight convolution module includes a channel splitting module, a two-dimensional convolution module and a channel splicing module; the input feature map is first split into feature map 1 and feature map 2 according to the proportion by the channel splitting module, and the feature map 1 is subjected to feature extraction by the two-dimensional convolution module and then channel spliced ​​with the feature map 2 to obtain the output feature map.

4. A helicopter blade vibration monitoring method based on deep learning dichotomy according to claim 3, characterized in that: The specific steps of step S3 are: Step S301: using the LabelImg annotation tool to annotate the marking points on the real-time blade vibration image collected in step S2, obtaining a corresponding label file, and combining the blade vibration image and the corresponding label file into a marking point detection data set; Step S302: training the improved YOLOv8n marker detection network, inputting the marker detection data set into the trained improved YOLOv8n marker detection network for recognition, and outputting the position information of the markers; Step S303: cropping a blade marker point image from the blade vibration image, and performing binary segmentation on the cropped marker point image to obtain a marker point binary image of the same size as the cropped blade marker point image, and forming a marker point segmentation data set with the blade marker point image and the corresponding marker point binary image; Step S304: construct an ellipse segmentation network; Step S305: training the ellipse segmentation network, using binary cross entropy loss as a loss function to calculate the segmentation error of the marker points during the training process, back-propagating the error, updating the network parameters, achieving network convergence, and continuously training the network until the error between the predicted marker point center coordinates and the true center coordinates is within 0.5 pixels, and based on the calculated segmentation error, saving a set of model parameters with the minimum loss value or the minimum average center error during the training process; Step S306: Load the ellipse segmentation network model and set the network's hyperparameters, and crop all circular marker point images from the blade vibration image according to the position information of the marker points obtained in step S302, and scale them to the same size as the cropped blade marker point images; then input them into the ellipse segmentation network for processing, the ellipse segmentation network outputs a binary image of each marker point and performs ellipse fitting on the binary image to obtain the center coordinates of all marker points; according to the scaling relationship when the marker points are cropped, the center coordinates are transformed into the original image to obtain the two-dimensional pixel coordinates of the center of each circular marker point in the original image.

5. A helicopter blade vibration monitoring method based on deep learning dichotomy according to claim 4, characterized in that: The calculation formula of the loss function is as follows: ； in, YP Represents the predicted value output by the ellipse segmentation network; YT Represents the true value corresponding to the input image; Loss ( YT, YP ) represents the loss function, that is, the segmentation error between the network prediction value and the true value yp ij Represents the predicted image i OK j Pixel values ​​on columns; yt ij Represents the real image i OK j The pixel value on the column.

6. A helicopter blade vibration monitoring method based on deep learning dichotomy according to claim 5, characterized in that: The specific steps of step S4 are: collecting a blade image in a static state as an initial state, using a circle center positioning method based on deep learning dichotomy to obtain the two-dimensional pixel coordinates of the center of all marking points in the initial state, and according to the mapping relationship between the two-dimensional pixel coordinates and the three-dimensional world coordinates, converting each blade marking point from the two-dimensional pixel coordinates of the circle center to the three-dimensional world coordinates as follows: XS k , YS k , 0) and save, where, k =1,2,..., n , n Indicates the number of marking points on the blade. XS k Indicates k The initial three-dimensional horizontal coordinates of the marker points, YS k Indicates k The initial three-dimensional vertical coordinates of the marker points.

7. A helicopter blade vibration monitoring method based on deep learning dichotomy according to claim 6, characterized in that: The specific steps of step S5 are: using an industrial camera to take a real-time image of the blade vibration, using a circle center positioning method based on deep learning dichotomy to obtain the two-dimensional pixel coordinates of the center of all marked points in the vibration image, and according to the mapping relationship between the two-dimensional pixel coordinates and the three-dimensional world coordinates, converting all the two-dimensional pixel coordinates of the center of the circle into three-dimensional world coordinates ( XE k , YE k , 0), where k =1,2,..., n , k Indicates the sequence number of the marking point. n Indicates the number of marking points on the blade. XE k Indicates k The three-dimensional horizontal coordinate of a marked point at a certain moment in the vibration process, YE k Indicates k The three-dimensional vertical coordinates of a marking point at a certain moment in the vibration process; the blade vibration amount at each marking point position is calculated h k =| YE k - YS k |, complete the calculation of the vibration amount at each position of the blade.

Images