A real-time monitoring method for outdoor helicopter tail rotor vibration based on YOLOV3-Tiny

Abstract

The present invention relates to a real-time monitoring method for the vibration of a helicopter tail rotor based on YOLOV3‑Tiny, which comprises the following steps: 1) computer vision calibration; 2) real-time tail rotor image acquisition; 3) intelligent positioning of the tail rotor tip area; 4) positioning of the aerodynamic center point of the tail rotor tip; 5) real-time monitoring of the vibration of the tail rotor. The present invention utilizes computer vision technology to perform high-speed, high-quality tail rotor image acquisition and computer vision calibration, and has the characteristics of high precision; adopts the YOLOV3‑Tiny deep learning network to intelligently position the tail rotor tip area, ensuring accurate positioning outdoors under complex lighting and backgrounds at all times, and has the characteristics of strong adaptability and high speed; locates the aerodynamic center point of the blade tip through the geometric characteristics of the blade tip, and realizes real-time monitoring of the vibration of the tail rotor. The present invention can be used for real-time monitoring of the vibration of the helicopter tail rotor, providing data support for the design and improvement of new helicopters, and indicating the direction of improvement.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision real-time monitoring, and in particular relates to a real-time monitoring method for outdoor helicopter tail rotor vibration based on a YOLOV3-Tiny target detection algorithm combined with computer vision technology. Background Art

[0002] The tail rotor is a core component of a helicopter's propulsion system, playing a crucial role in steering, overcoming rotational torque, enhancing maneuverability, and maintaining force balance. During flight, the reaction torque from the rotating rotor causes the helicopter to rotate in the opposite direction. The pull generated by the tail rotor counteracts this rotation, maintaining directional stability. Simultaneously, varying the tail rotor pull allows for directional control. Helicopter tail rotor vibration is a critical parameter of helicopter motion. Monitoring tail rotor vibration is important for two reasons: first, the complex low-altitude meteorological environment, characterized by high winds and turbulence, can easily lead to tail rotor failure, resulting in loss of control. Second, monitoring tail rotor forces during the design and testing of new helicopters is crucial to provide data support and guidance for design and improvement. Therefore, monitoring tail rotor vibration is of vital practical value for ensuring helicopter flight safety and developing new aircraft.

[0003] The core issues with tail rotor vibration monitoring are: first, when the tail rotor is rotating at high speed, its speed is much higher than that of the rotor, reaching over 2000 rpm. Therefore, the speed requirement for tail rotor vibration monitoring is very high. Second, because the experiment is conducted outdoors, real-time monitoring is required under all-weather conditions and complex lighting and background conditions. Tail rotor vibration measurement typically uses contact sensors, which offer fast monitoring speed and high accuracy. However, these sensors can only monitor the overall vibration of the tail rotor system, not the vibration of individual blades, and are easily affected by interference from other system components, making them inconvenient to operate. Helicopter rotor motion parameter measurement utilizes stereo vision, which offers high accuracy, is contactless, and simple to operate. However, these sensors are difficult to adapt to monitoring in complex outdoor environments. Therefore, research on real-time monitoring methods for outdoor helicopter tail rotor vibration has important theoretical research significance and practical application value. Summary of the Invention

[0004] To address the shortcomings of the existing technology, this paper provides a real-time monitoring method for outdoor helicopter tail rotor vibration based on YOLOV3-Tiny. By combining computer vision technology and deep learning networks, it can achieve rapid and accurate positioning of the tail rotor tip area, thereby achieving real-time monitoring of helicopter tail rotor vibration.

[0005] A real-time monitoring method for outdoor helicopter tail rotor vibration based on YOLOV3-Tiny includes the following steps:

[0006] Step (1) computer vision calibration;

[0007] Step (2) real-time tail rotor image acquisition;

[0008] Step (3) Intelligent positioning of the tail rotor tip area, the specific steps are as follows:

[0009] Step (3.1) Image dataset preparation. First, build a tail rotor simulation experimental environment; second, use the LabelImg annotation tool to annotate the tail rotor tip image and extract the category and position information of the target in the image; then, perform a matching check on the annotated image and position information; finally, convert the position relationship into a text file, and put the corresponding tail rotor image and position information file into two folders with different names in the same directory. All the collected tail rotor images and the obtained tail rotor tip position information files together constitute the image dataset, and the image dataset is divided into a training set and a test set.

[0010] Step (3.2) Training set data enhancement. First, the HSV color space adjustment method is used to expand the diversity of the data to adapt to complex lighting environments. Second, horizontal flipping and mosaic stitching are used to increase the complexity of the data to adapt to complex situations with different backgrounds.

[0011] Step (3.3) Network construction and training.

[0012] Step (3.3.1) Network construction. First, configure the software environment required for network construction and training. The network structure consists of three parts: feature extraction network D1, first output network D2, and second output network D3. Feature extraction network D1 contains an input layer, and a structure that alternates five convolutional layers and four pooling layers. Both the first output network D2 and the second output network D3 contain two-dimensional convolutional layers with convolution kernel sizes of 3x3 and 1x1. Feature extraction network D1 is used to extract image features; the first output network D2 and the second output network D3 are used to output two feature maps of different sizes.

[0013] Step (3.3.2) preset the size of the network candidate box;

[0014] Step (3.3.3) Network training: First, set the network data input path; second, preset the network hyperparameter information; finally, perform training. During the training process, the network model uses the CIoU indicator to calculate the coordinate error of the candidate box, uses BCEWithLogitsLoss (binary cross entropy) to calculate the confidence error and classification error, and calculates the gap between the image dataset data / training set data and the network model output. The calculation formula is:

[0015]

[0016] Where IOU is the intersection-of-union ratio of the bounding box and the candidate box; Distance_2 is the length of the diagonal of the bounding box; Distance_C is the Euclidean distance between the candidate box and the center point of the bounding box; v is an indicator to measure the consistency of the aspect ratio of the candidate box and the bounding box; y i is the training set data, x i is the model output.

[0017] According to the calculated confidence error and classification error data, save the minimum loss value or average accuracy (mAP) during training. 0.5 ) is the highest set of model parameters.

[0018] Step (3.4) YOLOV3-Tiny intelligent positioning. First, the model parameters are loaded and set; second, the network output is obtained; finally, the tail rotor tip area is intelligently located. The non-maximum suppression algorithm is used to remove overlapping candidate boxes from the network output. Then, a confidence threshold is used to eliminate data below the threshold. Finally, the target location and category information are obtained, achieving intelligent positioning of the tail rotor tip area.

[0019] Step (3.5) Model Engineering. First, re-import the model parameters obtained in step (3.3.3) into the network. Second, save the network structure and model parameters into a specific dictionary file. Then, convert the dictionary file into the ONNX general model framework. Finally, read the ONNX file into memory. After successful reading, the ONNX file can be used in the project.

[0020] Step (4) Positioning the aerodynamic center point of the tail rotor tip.

[0021] Step (5) Real-time monitoring of tail rotor vibration, the specific steps are as follows:

[0022] Step (5.1) Reference selection: During vibration monitoring, select the aerodynamic center point of the blade tip with the most stable vibration value among all blades as the reference point;

[0023] Step (5.2) Calculation of vibration amount. Divide the x value of the aerodynamic center point of the tail rotor tip obtained in step (4) by the x value of the reference point obtained in step (5.1), and multiply the difference by the calibration coefficient obtained in step (1) to obtain the vibration amount of the tail rotor. The calculation formula is:

[0024] val=(pt_n_new.x-criterion)*calibration

[0025] Where val is the vibration value, pt_n_new.x is the x-value of the aerodynamic center point of the blade tip, criterion is the x-value of the reference point, and calibration is the calibration coefficient.

[0026] Furthermore, the specific steps of computer vision calibration in step (1) are as follows:

[0027] Step (1.1) constructing an image acquisition device by using a processing computer, a signal triggering device, an industrial camera, a strobe light, and a photoelectric sensor; connecting the processing computer to the signal multiplier via a Gigabit Ethernet interface;

[0028] Step (1.2) uses a processing computer to search for and connect the industrial camera and the signal multiplier;

[0029] Step (1.3) Place the calibration template in the field of view of the industrial camera, set the target parameters, and use the industrial camera to capture an image of the calibration template;

[0030] Step (1.4) collects multiple sets of images and uses the ratio between the calibration template and the size of the collected images to obtain the calibration parameter calibration, which is calculated as follows:

[0031]

[0032] Where R d is the center distance of the calibration template, S p is the number of pixels between the centers of the circles.

[0033] Furthermore, the specific steps for positioning the aerodynamic center point of the tail rotor tip in step (4) are as follows:

[0034] Step (4.1) Tip Contour Extraction: Perform contour detection on the tip area output by the model to obtain a series of point sets. The point set with the largest number of points in the series is retained. Each point in this point set represents the pixel position of the tip contour.

[0035] Step (4.2) preliminarily locates the leftmost, rightmost, topmost, and bottommost points of the blade tip contour. Sort the points in the retained point set by x value from small to large, taking the first point after sorting as the leftmost point in the point set and the last point as the rightmost point in the point set; then sort the points in the point set by y value from small to large, taking the first point after sorting as the topmost point in the point set and the last point as the bottommost point in the point set. The calculation formula is as follows:

[0036]

[0037] Where pt_start is the leftmost point in the point set, pt_end is the rightmost point in the point set, pt_CoorUp is the topmost point in the point set, pt_CoorDown is the bottommost point in the point set, and point_set[i] is the point set with the largest number of points in a series of point sets contours.

[0038] Step (4.3) accurately locates the aerodynamic center point. First, find the coordinates of the quarter point close to the leftmost point in the point set in the line connecting the leftmost point and the rightmost point in the point set, and update the x value of the top and bottom points by the positional relationship between the quarter point and the top point in the point set and the line connecting the leftmost point and the rightmost point in the point set. The calculation formula is shown in the following formula (1); secondly, update the y value of the leftmost point by the positional relationship between the line connecting the updated top point and the updated bottom point and the line connecting the leftmost point and the rightmost point. The calculation formula is shown in the following formula (2); finally, take the quarter point close to the updated leftmost point in the line connecting the updated leftmost point and the rightmost point as the aerodynamic center point of the blade tip. The calculation formula for its coordinate value is shown in the following formula (3);

[0039]

[0040]

[0041] Where pt_start is the leftmost point in the point set, pt_end is the rightmost point in the point set, pt_n is the quarter point close to the leftmost point in the point set; pt_CoorUp_new is the updated uppermost point, pt_CoorDown_new is the updated lowermost point, pt_start_new is the updated leftmost point; pt_n_new.x represents the x value of the aerodynamic center point, and pt_n_new.y represents the y value of the aerodynamic center point.

[0042] The advantages of the real-time monitoring method for the vibration of the helicopter tail rotor of the present invention are: (1) The present invention adopts the YOLOV3-Tiny network to realize the intelligent positioning of the tail rotor tip area under complex lighting and background, with an accuracy rate of 99%, breaking through the limitations of traditional image processing methods and realizing the measurement of the vibration of the helicopter tail rotor in outdoor environments. (2) The present invention adopts computer vision calibration technology to realize non-contact measurement with a measurement accuracy of less than 1mm. (3) The present invention can meet the requirements of real-time monitoring at a tail rotor speed of about 2000rpm. The real-time monitoring method of the present invention combines the target detection algorithm of deep learning with computer vision technology, and can monitor the vibration of each blade of the helicopter tail rotor. It has the advantages of strong adaptability, high accuracy, non-contact, easy operation, and low interference. It can be applied to the tail rotor data detection during helicopter design, providing data support for the design and improvement of the helicopter and indicating the direction of improvement; it can also be applied to the monitoring and control of the tail rotor status of the helicopter during flight. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 4 is a flowchart of the real-time monitoring method in an embodiment of the present invention.

[0044] Figure 2 2 is a calibration diagram of the real-time monitoring method in an embodiment of the present invention; wherein (a) is the image collected for calibration, and (b) is the calibration result image.

[0045] Figure 3 3. It is a schematic diagram of real-time acquisition of blade tip images in the real-time monitoring method in an embodiment of the present invention.

[0046] Figure 4 2 is a YOLOV3-Tiny network structure diagram of the real-time monitoring method in an embodiment of the present invention.

[0047] Figure 5 It is a flow chart of the real-time monitoring method model engineering in an embodiment of the present invention.

[0048] Figure 6 1 is a diagram showing the positioning result of the tail rotor tip area according to the real-time monitoring method in an embodiment of the present invention; (a) and (b), (c) and (d), and (e) and (f) are three groups of detection result diagrams under different light intensities under the same background.

[0049] Figure 7 Schematic diagram of the geometric shape of the tail rotor tip in the real-time monitoring method according to an embodiment of the present invention.

[0050] Figure 8 2 is a schematic diagram of a method for real-time monitoring of the vibration of a tail rotor tip according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0052] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning understood by persons having ordinary skills in the field described in this application. The words "first", "second" and similar terms used in this patent application specification and claims do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship also changes accordingly.

[0053] Example:

[0054] The present embodiment is a real-time monitoring method for outdoor helicopter tail rotor vibration based on YOLOV3-Tiny, and its working process is as follows: Figure 1 As shown, the following steps are included:

[0055] Step (1) Computer vision calibration. Figure 3 As shown, this embodiment constructs an image acquisition device by using a processing computer, a signal triggering device, an industrial camera, a strobe light, and a photoelectric sensor. In this embodiment, the processing computer CPU used is Intel (R) Core (TM) i9-8950HK, the GPU is GTX 1060, the video memory is 6GB, the internal memory is 16GB, and the operating system is WIN10; the signal triggering device uses a signal multiplier (an oscilloscope can also be used instead); a German IDS digital industrial camera and a 25mm Japanese TV LENS lens are used to collect the target image of the calibration template; the circular marking points of the target in the calibration template are 7 rows and 8 columns, with a center distance of 42.00mm, and the size of the collected image is fixed to 1280x1024. The calibration template is kept within the camera's field of view, and the ratio between the image pixels and the actual length, i.e., the calibration coefficient, is calculated, which includes the following steps:

[0056] Step (1.1) connects the processing computer to the signal multiplier via a Gigabit Ethernet interface;

[0057] Step (1.2) uses a processing computer to search for and connect the industrial camera and the signal multiplier;

[0058] Step (1.3) Place the calibration template in front of the industrial camera, keep the calibration template within the field of view of the industrial camera, input the number of rows, columns, and center distance of the targets in the calibration template into the processing computer, and use the industrial camera to capture the target image of the calibration template;

[0059] In step (1.4), the calibration method using the EDCircles circular calibration plate is used on the processing computer to obtain the calibration coefficient. The calculation formula for the calibration coefficient is as follows:

[0060]

[0061] Where R d is the center distance of the calibration template, S p is the number of pixels between the two circle centers. Figure 2 .

[0062] Step (2) Real-time tail rotor image acquisition. In this embodiment, an industrial fan is used to simulate the helicopter tail rotor, a K48 strobe light with an angle of 5 degrees is used to fill in the tail rotor light, and a photoelectric sensor with model SM312LVMHSQD is used to generate a trigger signal. The photoelectric sensor is fixed to the side of the tail rotor blade through a bracket, and a highly reflective reflective sticker is attached to the tip of the tail rotor to ensure that the industrial camera can better capture the tip of the tail rotor. When the helicopter tail rotor blade rotates to the front of the camera, the photoelectric sensor located to the side of the tail rotor blade triggers the industrial camera to capture the image of the tail rotor with the reflective sticker attached to the tip, and at the same time triggers the strobe light to flash synchronously to fill in the tail rotor light. When each blade rotates to the front of the industrial camera, it will trigger the photoelectric sensor on the side to generate a trigger signal. The signal will be processed by the computer and transmitted to the industrial camera and the strobe light, thereby triggering the industrial camera to capture the image and the strobe light to flash. Based on this, the trigger signals can be numbered sequentially, and the number is the remainder of the number of blades. In this embodiment, the number of blades is 3. After taking the remainder of the number, the number with a remainder of 0 corresponds to the image of blade A0, the number with a remainder of 1 corresponds to the image of blade A1, and the number with a remainder of 2 corresponds to the image of blade A2, thereby realizing the recognition of the tail rotor image.

[0063] Step (3) Intelligent positioning of the tail rotor tip area. This includes the following steps:

[0064] Step (3.1) Image dataset creation. First, build a tail rotor simulation experimental environment using Figure 3The image acquisition device shown collects tail rotor images of various scenes to obtain a sample set; secondly, the LabelImg annotation tool is used to annotate the tail rotor tip images in the sample set, and the position of the tail rotor tip area in the tail rotor image is marked, and a label is set for the tip. After the annotation, the tool can automatically generate a position information file of the tail rotor tip area; then, a one-to-one matching check is performed on the tail rotor images and the position information files, and the corresponding tail rotor images and position information files are respectively placed in two folders with different names in the same level directory to prevent the missing tail rotor images from being added to the image data; finally, all the collected tail rotor images and the obtained tail rotor tip position information files together constitute an image dataset, and then the image dataset is randomly divided into a training set and a test set in a ratio of 4:1.

[0065] Step (3.2) Training set data enhancement. First, the HSV color space adjustment method is used to adjust the hue (H), saturation (S), and brightness (V) of the collected tail rotor image to adapt to complex lighting environments. This method is an existing method in the field of data enhancement. In this embodiment, H = 0.5, S = 0.5, and V = 0.5. Secondly, horizontal flipping and mosaic stitching are used. Horizontal flipping is to mirror the image data, and mosaic stitching is to randomly crop four images and reassemble them into four images. Both are commonly used data enhancement methods in deep learning and can increase data complexity to adapt to complex situations with different backgrounds.

[0066] Step (3.3) Network construction and training.

[0067] Step (3.3.1) Network construction. First, configure the software environment required for network construction and training. The network model structure of this embodiment can be found in Figure 4 This network model is improved on the basis of the traditional YOLOV3-Tiny model. Its structure consists of three parts: feature extraction network D1, first output network D2, and second output network D3. The feature extraction network D1 contains an input layer, and a structure of five convolutional layers and four pooling layers alternating. The first output network D2 and the second output network D3 both contain two-dimensional convolutional layers with convolution kernel sizes of 3x3 and 1x1. Figure 4As shown in the figure, "Conv" represents the convolutional layer, "Conv 5*5 / 2" indicates that the convolution kernel size in this convolutional layer is 5x5 and the stride of the convolution kernel is 2, "(batch_size,416,416,16)" represents the data size of the output of this layer, batch_size indicates the number of images input to the network at a time, (416,416,16) indicates that the output feature data is 416x416 in size and has 16 channels, and "Maxpool" represents the maximum pooling layer. The convolutional layer extracts image features such as texture and gradient through convolution operations, without changing the size of the input data but changing the number of channels, in order to extract as many image features as possible. The pooling layer filters the extracted features through maximum pooling, retaining the most significant ones. This allows for more accurate extraction of the tail rotor tip target's position information. Furthermore, the single alternating structure significantly improves the model's speed.

[0068] Compared with YOLOV3-Tiny, the detection method of the present invention improves the convolution kernel size to 5x5 in the second layer of the network, and the pooling area of the third maximum pooling layer to 4x4. The convolution layer of the fourth layer and the maximum pooling layer of the fifth layer in the original network are deleted, and the number of convolution kernels in the sixth layer is increased to 64, which is twice that of the original network. This can avoid excessive interference from complex backgrounds and reduce the network depth, thereby speeding up the model speed. Secondly, the multi-scale fusion operation of the network head detection is deleted, and two feature maps are directly output with sizes of (26, 26, 18) and (13, 13, 18), respectively, which reduces the branches of the network and improves the speed of the network. Finally, the CIoU indicator is used to calculate the coordinate error of the prediction box, and BCEWithLogitsLoss (binary cross entropy) is used to calculate the confidence error and classification error, which can achieve more accurate positioning.

[0069] The enhanced training set from step (3.2) above is input into the network model, and image features are extracted through the feature extraction network D1. The image data of the training set is resized to (416, 416, 3) when entering the network model. This is the fixed size of the image data upon entry, ensuring that the data size at output does not vary due to different input image resolutions. Subsequently, through the alternating effects of four convolutional layers and three pooling layers, a data size of (26, 26, 256) is obtained, which serves as the input of the first output network D2. The image data of the training set is further subjected to one convolution and one pooling to a data size of (13, 13, 512), which serves as the input of the second output network D3. In the first output network D2 and the second output network D3, convolutions with kernel sizes of 3x3 and 1x1 are used for dimensionality reduction, which serve as the final two outputs of the network.

[0070] In step (3.3.2), the size of the network prior box is preset. The prior box is a pre-set box size used to help the model converge better. In the network model built in step (3.3.1), the sizes of the two feature maps outputted at the end are (26, 26, 18) and (13, 13, 18) respectively. During the detection process, the network divides the image data into grids for detection, and divides the image data into 26x26 and 13x13 grids for detection respectively. The image data outputted by the first output network D2 is divided into 26x26 grids for a total of 676 grids, and 3 prior boxes are assigned to each of these 676 grids, with sizes of 81x82, 135x169, and 344x319 respectively; similarly, the image data outputted by the second output network D3 is divided into 13x13 grids for a total of 169 grids, and 3 prior boxes of sizes of 10x14, 23x27, and 37x58 are assigned to each of these 169 grids. In this way, 26x26x3+13x13x3 candidate boxes can be obtained, and the network model will select a box from these candidate boxes as the position of the final target (i.e., as the bounding box).

[0071] Step (3.3.3) Network training. First, set the training set and test set paths in step (3.1) as the data input of the network model; second, preset the hyperparameter information of the network model. In this embodiment, the number of training rounds is set to 1000, the learning rate is 1E-5, the momentum is 0.937, the number of warmup rounds is 3, and batch_size = 1; finally, training is performed. During the training process, the network model uses the CIoU indicator to calculate the coordinate error of the candidate box, uses BCEWithLogitsLoss (binary cross entropy) to calculate the confidence error and classification error, and calculates the gap between the image dataset data and the network model output. The calculation formula is:

[0072]

[0073] Where IOU is the intersection-of-union ratio of the bounding box and the candidate box; Distance_2 is the length of the diagonal of the bounding box; Distance_C is the Euclidean distance between the candidate box and the center point of the bounding box; v is an indicator to measure the consistency of the aspect ratio of the candidate box and the bounding box; y i is the training set data, x i is the model output.

[0074] After 1000 training rounds, the minimum loss value or average accuracy (mAP) during the training process is saved based on the calculated confidence error, classification error and coordinate error. 0.5 ) is the set of model parameters with the highest error (i.e., the model parameters with the smallest error value).

[0075] Step (3.4) YOLOV3-Tiny intelligent positioning.

[0076] Step (3.4.1) Model parameter loading and parameter setting. After obtaining the optimal set of model parameters through step (3.3.3), load the model parameters through Pytorch's load function and import the model parameters into the network model built in step (3.3). Set the confidence threshold to 0.5 and the IOU threshold to 0.5.

[0077] Step (3.4.2) obtains the network output. A single-channel image of 1280x1024x1 is used as input data and fed into the network model configured in step (3.4.1) with batch_size = 1. First, the image data is reduced to 416x416x3 in size after passing through the first layer of the feature extraction network D1. Then, after passing through subsequent convolutional and pooling layers, a first feature map B1 of 26x26x256 and a second feature map B2 of 13x13x512 are obtained. The first feature map B1 is then reduced in dimension by convolution with a 1x1 kernel in the first output network D2, and the second feature map B2 is further reduced in dimension by convolution with a 3x3 kernel in the second output network D3. The outputs are a third feature map B3 of 26x26x18 and a fourth feature map B4 of 13x13x18, respectively.

[0078] Step (3.4.3) Position the tail rotor tip area. First, the size of the third feature map B3 obtained in step (3.4.2) is adjusted to 26x26x3x6, and the size of the fourth feature map B4 is adjusted to 13x13x3x6. At this time, the network model obtains 26x26x3+13x13x3=2535 candidate boxes; then, the non-maximum suppression algorithm is used on these 2535 candidate boxes. This method is an algorithm for target detection to solve the overlap of candidate box areas and is widely used. The algorithm outputs an Nx6 tensor, where the value of N is unknown and the value of N is related to the IOU threshold in step (3.4.1); the tensor is a two-dimensional matrix data, and the data in its fifth column is compared one by one with the confidence threshold set in step (3.4.1). For data less than the threshold, the entire row of data in which the data is located is deleted, and data greater than the threshold is not processed; finally, a 1x6 tensor is obtained, which represents the position and category information of the target. By outputting it, the data information of the tip area position can be obtained, realizing the intelligent positioning of the tail rotor tip area.

[0079] Step (3.5) Model Engineering. First, re-import the model parameters obtained in step (3.3.3) into the network model, which is the network in step (3.3.3); secondly, save the network structure and model parameters into a dictionary file; then, convert the dictionary file into the ONNX general model framework to obtain the ONNX file; finally, use the readNet function of the OpenCV dnn library to read the ONNX file. After successful reading, it can be called in the C++ project. The model engineering process can be seen in Figure 5 .

[0080] Step (4) calculating the aerodynamic center of the tail rotor tip includes the following steps:

[0081] Step (4.1) Extract the blade tip contour. Use OpenCV's findContours function to perform contour detection on the blade tip area location data obtained in step (3.4.3). Multiple point sets (contours) are obtained. The point set with the largest number of points in the contours is retained and named point_set. The data in this point_set is the pixel position of the blade tip on the image.

[0082] Step (4.2) Locate the leftmost, rightmost, topmost, and bottommost points of the contour. First, sort the points in the point set point_set by their x values from small to large, taking the first point after sorting as the leftmost point in the point set pt_start, and the last point as the rightmost point in the point set pt_end; second, sort the points in the point set by their y values from small to large, taking the first point after sorting as the topmost point in the point set pt_CoorUp, and the last point as the bottommost point in the point set pt_CoorDown. The calculation formula is as follows:

[0083]

[0084] Where pt_start is the leftmost point in the point set, pt_end is the rightmost point in the point set, pt_CoorUp is the topmost point in the point set, pt_CoorDown is the bottommost point in the point set, and point_set[i] is the point set with the largest number of points in a series of point sets contours.

[0085] Step (4.3) Aerodynamic center point positioning. First, calculate the coordinates of the quarter point pt_n on the line connecting pt_start and pt_end, which is close to pt_start. Since the line connecting pt_n and pt_CoorUp must be perpendicular to the line connecting pt_start and pt_end, the following formula (1) can be obtained. From formula (1), the coordinates of pt_CoorUp_new and pt_CoorDown_new can be obtained. Secondly, since the line connecting pt_CoorUp_new and pt_CoorDown_new must be perpendicular to the line connecting pt_start and pt_end, the following formula (2) can be obtained. From formula (2), pt_start_new can be obtained. Finally, the quarter point on the line connecting pt_start_new and pt_end, which is close to pt_start_new, is taken as the aerodynamic center point pt_n_new. The coordinate value calculation formula of pt_n_new is shown in the following formula (3).

[0086]

[0087] Where, pt_CoorUp_new is the updated uppermost point, pt_CoorDown_new is the updated lowermost point, pt_start_new is the updated leftmost point; pt_n_new.x represents the x value of the aerodynamic center point, and pt_n_new.y represents the y value of the aerodynamic center point. Figure 7 .

[0088] Step (5) real-time monitoring of the tail rotor vibration includes the following steps.

[0089] Step (5.1) Reference selection: During the vibration monitoring process, the aerodynamic center point of the blade tip with the most stable vibration value among all blades is selected as the reference point.

[0090] Step (5.2) Calculation of vibration amount. Divide the x value of the aerodynamic center point of the tail rotor tip obtained in step (4) by the x value of the reference point obtained in step (5.1), and multiply the difference by the calibration coefficient obtained in step (1) to obtain the vibration amount of the tail rotor. The calculation formula is:

[0091] val=(pt_n_new.x-criterion)*calibration

[0092] Where val is the vibration amount, pt_n_new.x is the x-value of the aerodynamic center point of the blade tip, criterion is the x-value of the reference point, and calibration is the calibration coefficient. For a schematic diagram of the tail rotor tip vibration amount, see Figure 8 .

[0093] In the hardware environment shown in Table 1 below, the robustness, positioning accuracy, and positioning time of the outdoor helicopter tail rotor vibration real-time monitoring method based on YOLOV3-Tiny were tested, and the results are as follows.

[0094] Table 1 Experimental hardware platform

[0095] equipment Model parameters CPU Intel(R)core(TM)I9-8950HK@2.90GHZ GPU NVIDIA GeforceGTX1060withMax-QDesgin6GB Memory Samsung 16GB disk SKhynixSC311SATA

[0096] 1. Robustness test.

[0097] The industrial fan simulating the tail rotor of a helicopter was tested under various lighting and background conditions, such as Figure 6 As shown, each image is a single-channel image captured using the same experimental equipment. The different grayscale values of each image are caused by the intensity of external light. The experiment was conducted under two different light intensities and three different backgrounds. Figures (a) and (b) are the experimental results under the same outdoor background, but with different light intensities at noon and afternoon. Figures (c) and (d), and (e) and (f) are all detection results under the same outdoor background but with different lighting conditions. In these six outdoor environments, the model can effectively locate the tip of the tail rotor. The experiments show that the model overcomes the influence of complex lighting and background and can adapt well to complex outdoor environments.

[0098] 2. Positioning accuracy test.

[0099] 2.1) During the network training process in step (3.3), the training results are shown in Table 2.

[0100] Table 2 Model training indicator data

[0101]

[0102] During network training, 6 sets of data are averaged from 0 to 500 training data. Precision and Recall evaluate the accuracy of classification from different perspectives. The maximum value is 1. The higher the value, the more accurate the classification. mAP 0.5 and mAP 0.5:0.95 Both evaluate the similarity between the candidate box and the bounding box, with a maximum value of 1, and a larger value indicates better target detection. As can be seen from the data in Table 2, each metric increases with the number of training data points and approaches the maximum value of 1, indicating that the model can accurately locate the blade tip region in almost all the tail rotor images tested, demonstrating relatively accurate positioning.

[0103] 2.2) After step (3.3), the optimal model parameters are obtained and tested. The experimental results are shown in Table 3.

[0104] Table 3 Figure 6 Bounding box data of each image

[0105]

[0106] a~f correspondence Figure 6 In each figure, width and height represent the width and height of the bounding box. Since the size of the detected tail rotor tip target is fixed, its aspect ratio is constant, which is 3.0. Based on the accurate positioning of the positioning accuracy test 2.1), the positioning accuracy can be further evaluated based on whether the aspect ratio of the bounding box is consistent with the aspect ratio of the tip. The values of width and height in the table are the number of image pixels, which are the results calculated based on computer vision calibration and bounding box information. The positioning accuracy is accurate to the pixel. As can be seen from Table 3, when the measurement distance is constant, the width and height of each bounding box in each group of experiments are basically the same, indicating that the model has overcome the influence of complex lighting and the positioning accuracy is not affected by lighting; for all scenes a~f, the aspect ratio of the bounding box is always concentrated around 3.0, indicating that the model has overcome the influence of complex lighting and background and achieved high-precision positioning of the tail rotor tip area.

[0107] 3. Positioning Time Test. Calculated from the maximum operating speed of the helicopter's tail rotor, the total time for each image acquisition, processing, saving processed data, and visualizing processed data must be strictly controlled below 14ms, and the processing frame rate must be above 70fps. The experimental time data is shown in Table 4 below.

[0108] Table 4 Positioning time

[0109] Equipment Environment speed GTX1060 6GB 15ms GTX1080Ti8GB 4ms

[0110] With the GTX 1060 GPU, the model call speed is 15ms. With the help of multi-threading technology, the series of operations for each image acquisition, processing, data storage, and data visualization can be controlled below 14ms. With the GTX 1080Ti GPU, the call speed is 4ms, which fully meets the real-time monitoring requirements of the project.

[0111] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A real-time monitoring method for outdoor helicopter tail rotor vibration based on YOLOV3-Tiny, characterized in that: The following steps are involved: Step (1) Computer vision calibration; Step (2) real-time tail rotor image acquisition; Step (3) Intelligent positioning of the tail rotor tip area. The specific steps are as follows: Step (3.1) Image dataset preparation: First, build a tail rotor simulation experimental environment; second, use the LabelImg annotation tool to annotate the tail rotor tip image and extract the category and position information of the target in the image; then, perform a matching check on the annotated image and position information; finally, convert the position relationship into a text file, and place the corresponding tail rotor image and position information files into two folders with different names in the same directory. All the collected tail rotor images and the obtained tail rotor tip position information files together constitute the image dataset, and the image dataset is divided into a training set and a test set. Step (3.2) Training set data enhancement: First, the HSV color space adjustment method is used to expand the diversity of the data to adapt to complex lighting environments; second, horizontal flipping and mosaic splicing are used to increase the complexity of the data to adapt to complex situations with different backgrounds; Step (3.3) Network construction and training, the specific steps are as follows: Step (3.3.1) Network Construction: First, configure the software environment required for network construction and training. The structure consists of three parts: feature extraction network D1, first output network D2, and second output network D3. Feature extraction network D1 consists of an input layer and a structure that alternates five convolutional layers and four pooling layers. Both the first output network D2 and the second output network D3 contain two-dimensional convolutional layers with convolution kernel sizes of 3x3 and 1x1. Feature extraction network D1 is used to extract image features. The first output network D2 and the second output network D3 are used to output two feature maps of different sizes. Step (3.3.2) preset the size of the network candidate box; Step (3.3.3) Network training: First, set the network data input path; second, preset the network hyperparameter information; finally, perform training. During the training process, the network model uses the CIoU indicator to calculate the coordinate error of the candidate box, uses BCEWithLogitsLoss (binary cross entropy) to calculate the confidence error and classification error, and calculates the gap between the image dataset data / training set data and the network model output. The calculation formula is: ； Where IOU is the intersection-of-union ratio of the bounding box and the candidate box; Distance_2 is the length of the diagonal of the bounding box; Distance_C is the Euclidean distance between the candidate box and the center point of the bounding box; v is an indicator to measure the consistency of the aspect ratio of the candidate box and the bounding box; y i is the training set data, x i is the model output; Based on the calculated confidence error, classification error, and coordinate error data, save the set of model parameters with the minimum loss value or the highest mean average accuracy (mAP@0.5) during training; Step (3.4) YOLOV3-Tiny intelligent positioning: First, load and set the model parameters; second, obtain the network output; finally, intelligently locate the tail rotor tip area. For the network output, use the non-maximum suppression algorithm to remove overlapping candidate boxes, then use the confidence threshold to eliminate data below the threshold one by one. Finally, obtain the target's location and category information, and realize intelligent positioning of the tail rotor tip area. Step (3.5) Model Engineering: First, re-import the model parameters obtained in step (3.3.3) into the network; second, save the network structure and model parameters into a dictionary file; then, convert the dictionary file into the ONNX general model framework; finally, read the ONNX file into memory. After successful reading, the ONNX file can be put into the project for use; Step (4) positioning the aerodynamic center point of the tail rotor tip; Step (5) Real-time monitoring of tail rotor vibration. The specific steps are as follows: Step (5.1) Reference selection: After the vibration monitoring begins, select the aerodynamic center point of the blade tip where the vibration value is most stable within 10 minutes of the monitoring start as the reference point; Calculate the vibration amount in step (5.2). Divide the x value of the aerodynamic center point of the tail rotor tip obtained in step (4) by the x value of the reference point obtained in step (5.1), and multiply the difference by the calibration coefficient obtained in step (1) to obtain the vibration amount of the tail rotor. The calculation formula is: ； Where val is the vibration value, pt_n_new.x is the x-value of the aerodynamic center point of the blade tip, criterion is the x-value of the reference point, and calibration is the calibration coefficient.

2. The outdoor helicopter tail rotor vibration real-time monitoring method based on YOLOV3-Tiny according to claim 1 is characterized in that, The specific steps of computer vision calibration in step (1) are as follows: Step (1.1) constructing an image acquisition device by using a processing computer, a signal triggering device, an industrial camera, a strobe light, and a photoelectric sensor; connecting the processing computer to the signal multiplier via a Gigabit Ethernet interface; Step (1.2) uses a processing computer to search for and connect the industrial camera and the signal multiplier; Step (1.3) Place the calibration template in the field of view of the industrial camera, set the target parameters, and use the industrial camera to capture an image of the calibration template; Step (1.4) collects multiple sets of images and uses the ratio between the calibration template and the size of the collected images to obtain the calibration coefficient. The calculation method is: ； Where R d is the center distance of the calibration template, S p is the number of pixels between the center points of the circles.

3. The outdoor helicopter tail rotor vibration real-time monitoring method based on YOLOV3-Tiny according to claim 1 is characterized in that, The specific steps for positioning the aerodynamic center point of the tail rotor tip in step (4) are as follows: Step (4.1) Extract the blade tip contour. Perform contour detection on the blade tip area output by the model to obtain a series of point sets, represented by the symbol contours. The point set with the largest number of points in the series, point_set, is retained. Each point in this point set represents the pixel position of the blade tip contour. Step (4.2) preliminarily locates the leftmost, rightmost, topmost, and bottommost points of the blade tip profile. Sort the points in point_set by x value from small to large, and take the first point after sorting as the leftmost point of the blade tip profile, and the last point as the rightmost point of the blade tip profile. Finally, sort the points in the point set by y value from small to large, and take the first point after sorting as the topmost point of the blade tip profile, and take the last point as the bottommost point of the blade tip profile. The calculation formula is as follows: ； Where pt_start is the leftmost point in the point set, pt_end is the rightmost point in the point set, pt_CoorUp is the topmost point in the point set, pt_CoorDown is the bottommost point in the point set, and point is any node in point_set. Its value consists of x and y, expressed as point.x and point.y. Step (4.3) accurately locates the aerodynamic center point. First, find the coordinates of the quarter point close to the leftmost point in the line connecting the leftmost and rightmost points. Based on the positional relationship between the quarter point and the top point and the line connecting the leftmost and rightmost points, update the x values of the top and bottom points. The calculation formula is shown in the following formula (1); Secondly, based on the positional relationship between the line connecting the updated uppermost point and the updated lowermost point and the line connecting the leftmost point and the rightmost point, the y value of the leftmost point is updated. The calculation formula is shown in the following formula (2); finally, the quarter point of the line connecting the updated leftmost point and the rightmost point close to the updated leftmost point is taken as the aerodynamic center point of the blade tip. The calculation formula of its coordinate value is shown in the following formula (3); (1) (2) (3)； Where pt_n is the quarter point close to the leftmost point in the point set, pt_CoorUp_new is the updated uppermost point, pt_CoorDown_new is the updated lowermost point, pt_start_new is the updated leftmost point, and pt_n_new is the aerodynamic center point.

Images