Coaxial rotor blade spacing real-time monitoring method in complex environment based on MB network

Abstract

The invention relates to a method for monitoring the distance between coaxial rotor blades in real time in a complex environment based on an MB network. The method comprises the following steps: installing an industrial camera and carrying out machine vision calibration; acquiring a blade tip image; a Mobile-Blade network model is constructed, and model training and deployment are carried out; and positioning a propeller tip reference point and calculating the propeller spacing. The method can effectively solve the problems that in an existing method, the aerodynamic characteristic influence of the blades is large, the detection precision of the blade tip mark points is low, and the detection speed is low, real-time high-precision monitoring of the coaxial rotor blade distance can be achieved in the complex environment, and data support is provided for design and improvement of a novel coaxial helicopter.

Description

Technical Field

[0001] The present application relates to the technical field of coaxial rotor blade pitch measurement, and in particular to a method for real-time monitoring of coaxial rotor blade pitch in a complex environment based on a MB network. Background Art

[0002] Coaxial rotor helicopters, due to their compact structure and excellent performance, are the primary configuration for next-generation military helicopters. However, the hub drag of coaxial rotor helicopters accounts for over 50% of the total aircraft drag. Reducing the spacing between the upper and lower rotors is an effective way to reduce drag. However, rotor elastic deformation increases at high speeds, and too small a spacing can easily lead to the risk of blade tip collisions.

[0003] The core issues of real-time monitoring of coaxial rotor blade pitch are: (1) when the upper and lower blades of a coaxial rotor helicopter rotate at a high speed, and the monitoring equipment needs to minimize the impact on the aerodynamic characteristics of the blades, traditional contact measurement methods are difficult to implement; (2) non-contact measurement methods based on traditional image processing methods cannot adapt to outdoor all-weather (i.e., morning, noon, evening, spring, summer, autumn, and winter) and complex lighting environments (i.e., front light, back light, and side light) during outdoor experiments, and are difficult to meet the real-time monitoring requirements in terms of adaptability, detection accuracy, and detection speed. Currently, measurement methods that combine general detection networks such as the mainstream DETR, MobileNet series, and YOLO series are gradually emerging. However, the anchor frame mechanism, multi-scale feature fusion strategy, and receptive field design of the general detection network of this type of method are mainly targeted at conventional scale targets. The detection accuracy and time for the specific shape of the blade tip marker still cannot meet the actual real-time monitoring requirements of blade pitch. Summary of the Invention

[0004] The purpose of the present invention is to provide a real-time monitoring method for coaxial rotor blade pitch in complex environments based on MB network, which can effectively solve the problems of large influence of blade aerodynamic characteristics, low detection accuracy of blade tip marker points, and slow detection speed in existing methods. It can realize real-time and high-precision monitoring of coaxial rotor blade pitch in complex environments, and provide data support for the design and improvement of new coaxial helicopters.

[0005] The technical solution adopted by the present invention is: a method for real-time monitoring of coaxial rotor blade pitch in a complex environment based on MB network, comprising the following steps: S1: Install an industrial camera at the same horizontal plane as the intersection center of the coaxial rotor blade tips and perform machine vision calibration; S2: Acquire blade tip image; S3: Build a Mobile-Blade network model, and perform model training and deployment; the Mobile-Blade network model includes a backbone network, a neck network, and a head network; the backbone network includes a lightweight bottleneck module for extracting features from the input image to obtain intermediate features and intermediate features The lightweight bottleneck module includes a 3×3 convolution layer, a batch normalization layer, and a ReLU activation layer; the neck network includes a lightweight feature fusion module for intermediate features. and intermediate features Perform feature fusion to obtain fusion features and fusion features The lightweight feature fusion module includes a 1×1 convolution layer, an upsampling layer, and a downsampling layer; the head network includes two lightweight detection head modules for fusion feature and fusion features Perform feature decoupling to obtain the confidence, category, and border of the paddle tip area. The lightweight detection head module includes a depth-separable convolution layer, a 1×1 convolution layer, and a two-dimensional convolution layer. S4: Locate the blade tip reference point and perform blade pitch calculation.

[0006] Furthermore, the industrial camera is installed at the same horizontal plane as the intersection center of the optical axis and the coaxial rotor blade tip, ensuring that its imaging center is at the same horizontal height as the intersection center point of the coaxial rotor blade tip motion trajectory.

[0007] Furthermore, the specific method of the machine vision calibration is: Use a circular calibration template placed at the vertical plane where the coaxial rotor blade tips meet, set the target parameters, and use an industrial camera to collect k circular calibration template images; use the EDCircle algorithm to process each circular calibration template image to obtain the center coordinates and radius of all circular marking points; connect the centers of the circular marking points at the four corners of each circular calibration template image to form a quadrilateral, and use the line connecting the centers of the circular marking points at the upper right corner and the lower left corner as the diagonal to divide the quadrilateral into two triangles; use the machine vision calibration formula to solve and obtain the pixel equivalent result of the machine vision calibration , the specific expression of the machine vision calibration formula is: ; in, 、 and Represents the three side lengths of the i-th triangle in the j-th calibration template image, ; represents the semi-perimeter of the i-th triangle in the j-th calibration template image; Represents the area of ​​the i-th triangle in the j-th calibration template image; Indicates the center distance of the calibration template, Indicates the number of rows in the calibration template. Indicates the number of columns of the calibration template; Indicates the total number of calibration template images collected; represents the pixel equivalent of the j-th calibration template image, j = 1, 2, ..., k; Indicates the pixel equivalent result of machine vision calibration.

[0008] Furthermore, the specific process of collecting the blade tip image is: sticking a marker with reflective properties on the blade tip and illuminating the marker with a strobe light, and then using an industrial camera to collect the blade tip image.

[0009] Furthermore, the training and deployment process of the Mobile-Blade network model is as follows: collecting blade tip images in a real environment and a virtual environment of the software, annotating the collected blade tip images using an annotation tool to obtain corresponding label files, and forming a blade tip image dataset with the blade tip images and the corresponding label files; using the blade tip image dataset to train the Mobile-Blade network model to obtain the optimal model weight file; converting the optimal model weight file into a sequence file in .engine format and deploying the sequence file through the TensorRT platform.

[0010] Furthermore, the specific steps for locating the blade tip reference point are as follows: S401: The blade tip image collected in step S2 is detected by the Mobile-Blade network model to obtain all blade tip detection frame information in the image. Each blade tip detection frame information includes the X-axis coordinate and Y-axis coordinate of the upper left corner of the detection frame, the width W and the height H of the detection frame; and it is determined whether the number of blade tip detection frames is equal to ,in, Indicates the number of propeller tips that need to be detected correctly. , if the number of propeller tip detection frames is not equal to Then return to step S2 and re-collect the blade tip image. If the number of blade tip detection frames is equal to Then proceed to the next step; S402: Keep the center position of each blade tip detection frame unchanged and expand the area ,in , and set the grayscale value outside the blade tip detection frame area to 0 to obtain an extended mask image; perform Otsu threshold segmentation algorithm on the blade tip detection frame area in the extended mask image to obtain a threshold segmentation image; perform morphological closing operation on the threshold segmentation image to obtain a morphological closing operation image; extract all blade tip contours in the morphological closing operation image, and sort them from large to small according to the blade tip contour area, and output the n blade tip contours with the largest area, where ; S403: The blade tip profiles with the minimum and maximum Y-axis coordinates are respectively used as the upper blade tip and lower blade tip profiles; the starting point and the end point of the upper blade tip and the lower blade tip are connected, and the point where the connecting line is three-quarters of the distance from the end point is set as the aerodynamic center; the starting point and the end point of the upper blade tip and the lower blade tip are respectively corrected for errors with the aerodynamic center to obtain the blade tip reference point Tip reference point, , blade tip reference point , blade tip reference point , blade tip reference point and blade tip reference point .

[0011] Furthermore, the calculation formula for the propeller pitch is: ; in, represents the measured blade tip pitch, is the pixel equivalent result of machine vision calibration, Indicates the blade tip reference point , blade tip reference point or blade tip reference point The Y-axis coordinate of any reference point in Indicates the blade tip reference point , blade tip reference point or blade tip reference point The Y-axis coordinate of any reference point in .

[0012] The beneficial effects of the present invention are: the real-time monitoring system for blade pitch constructed by the present invention can realize non-contact measurement of coaxial rotor blade pitch; through the Mobile-Blade network model, rapid positioning of the blade tip in complex outdoor environments can be achieved with an accuracy of 99%. On this basis, image processing operations are performed to accurately locate the blade tip reference point coordinates of the upper and lower blades, and combined with the pixel equivalent results of machine vision calibration, the actual blade pitch is calculated with a measurement accuracy of less than 1mm; the average detection time per image of the present invention on the TensorRT platform is 3.2ms, and the maximum frame rate is 300FPS. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0014] Figure 1 is a flow chart of a method according to an embodiment of the present invention; Figure 2A schematic diagram of hardware device connections according to an embodiment of the present invention; Figure 3 A schematic diagram of machine vision calibration according to an embodiment of the present invention; Figure 4 Figure 1 is a schematic diagram of a blade tip image dataset according to an embodiment of the present invention; (a) is a blade tip image captured under direct sunlight at noon on a sunny day, (b) is a blade tip image captured under direct sunlight at afternoon on a sunny day, and (c) is a blade tip image captured under direct sunlight at noon on a cloudy day. Figure 5 A schematic diagram of the structure of a Mobile-Blade network model in an embodiment of the present invention; Figure 6 Schematic diagram of the structure of the lightweight bottleneck module of the Mobile-Blade network model in an embodiment of the present invention; Figure 7 Schematic diagram of the structure of the lightweight feature fusion module of the Mobile-Blade network model in an embodiment of the present invention; Figure 8 Schematic diagram of the structure of the lightweight detection head module of the Mobile-Blade network model in an embodiment of the present invention; Figure 9 This is a diagram showing the detection effect of the blade tip reference point in an embodiment of the present invention; Figure 10 Schematic diagram of the results of a generalization experiment under a simulated environment in an embodiment of the present invention; Figure 11 Schematic diagram of the average error of each group under a simulated environment in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0016] like Figure 1 As shown, the embodiment of the present invention provides a method for real-time monitoring of coaxial rotor blade pitch in a complex environment based on MB network. The hardware device connection and layout settings are as follows: Figure 2As shown, the hardware device includes a data processing unit 101, a signal triggering device 102, a switch 103, an industrial camera 104, a strobe light 105, a photoelectric sensor 106, and a coaxial rotor 107. The data processing unit 101, the signal triggering device 102, the switch 103, the industrial camera 104, the strobe light 105, and the photoelectric sensor 106 are electrically connected, and the blade tip image data of the coaxial rotor 107 collected by the industrial camera 104 can be transmitted to the data processing unit 101. The method for real-time monitoring of the coaxial rotor blade pitch in a complex environment based on the MB network includes the following steps: S1: Install industrial camera 104 and perform machine vision calibration. Industrial camera 104 is installed at the same horizontal plane as the intersection of the optical axis and the blade tip of coaxial rotor 107, ensuring that its imaging center is at the same horizontal height as the intersection of the blade tip motion trajectory of coaxial rotor 107. In this embodiment of the present invention, industrial camera 104 is model UI-5040CP-M-GLRev.2.2, with a camera lens focal length of 25mm. It is also equipped with a data processing unit 101, a signal trigger device 102, a switch 103, a strobe light 105, and a circular calibration template, forming a machine vision calibration device.

[0017] The specific method of the machine vision calibration is: After the machine vision calibration device is installed, the focal length and aperture size of the industrial camera 104 lens are adjusted to present a clear image of the blade tip; Use a circular calibration template to place it at the intersection of the coaxial rotor blade tip vertical plane, where the center distance of the circular mark point in the circular calibration template is 80.00mm, number of rows 6, number of columns The target parameters are set to 5, and k circular calibration template images are collected using an industrial camera. Each circular calibration template image is processed using the EDCircle algorithm to obtain the center coordinates and radius of all circular marking points. All circular marking points are sorted from top to bottom and from left to right and are marked as 1 to 30. In the embodiment of the present invention, the value of k is 3. For the jth circular calibration template image, the centers of the circular marking points at the four corners are connected to form a quadrilateral, and the quadrilateral is divided into two triangles using the line connecting the centers of the circular marking points at the upper right corner and the lower left corner as the diagonal line. The triangle at the top is denoted as S. 1j , the triangle at the bottom is denoted as S 2j , the first triangle S 1j The side lengths are a 1j 、b 1j and c 1j , the second triangle S 2j The side lengths are a 2j 、b 2jand c 2j , and then calculate the area of ​​the quadrilateral in this circular calibration template image; finally, use the machine vision calibration formula to solve and obtain the pixel equivalent result of the machine vision calibration ,like Figure 3 As shown, the specific expression of the machine vision calibration formula is: ; in, 、 and Represents the three side lengths of the i-th triangle in the j-th calibration template image, ; represents the semi-perimeter of the i-th triangle in the j-th calibration template image; Represents the area of ​​the i-th triangle in the j-th calibration template image; Indicates the center distance of the calibration template, Indicates the number of rows in the calibration template. Indicates the number of columns of the calibration template; Indicates the total number of calibration template images collected; represents the pixel equivalent of the j-th calibration template image, j = 1, 2, ..., k; Indicates the pixel equivalent result of machine vision calibration.

[0018] S2: Capture the blade tip image. The specific process is as follows: a marker with reflective properties is pasted on the blade tip and the marker is illuminated by a strobe light 105, and then the blade tip image is captured by an industrial camera 104. In an embodiment of the present invention, a photoelectric sensor 106 with model SM312LVMHSQD is used to generate a trigger signal. The photoelectric sensor 106 is fixed under the rotor, and a reflective sticker is pasted on the blade tip of the coaxial rotor 107 to ensure that the industrial camera 104 can better capture the blade tip image when the upper and lower rotors interact. When the coaxial rotor 107 rotates to the front of the industrial camera 104, the photoelectric sensor 106 located under the blade synchronously triggers the industrial camera 104 and the strobe light 105 to capture the blade tip image in real time. The real-time collected blade tip image is transmitted to the data processing unit 101 through the switch 103, and the trigger signal is sequentially numbered, and the number is the remainder of the number of blades. In an embodiment of the present invention, the number of blades is 4. After taking the remainder of the number, the number with a remainder of 0 corresponds to the image of blade A0, the number with a remainder of 1 corresponds to the image of blade A1, the number with a remainder of 2 corresponds to the image of blade A2, and the number with a remainder of 3 corresponds to the image of blade A3, thereby realizing the recognition of the blade image.

[0019] S3: Build the Mobile-Blade network model, namely the MB network model, and perform model training and deployment. Figure 5As shown, the Mobile-Blade network model includes a backbone network, a neck network, and a head network. The backbone network includes a lightweight bottleneck module for extracting features from the input image to obtain intermediate features. and intermediate features .like Figure 6 As shown, the lightweight bottleneck module includes a 3×3 convolution layer, a batch normalization layer, and a ReLU activation layer; the lightweight bottleneck module includes a left main path and a right branch path; the left main path contains two blocks, each block first performs a convolution operation on the input image with a step size of 1 and a convolution kernel of 3×3, then passes through a batch normalization layer and a ReLU activation layer for processing; the right branch processes the input image through a batch normalization layer and a ReLU activation layer, and performs a matrix addition operation with the output result of the left main path to obtain the output result of the lightweight bottleneck module. The neck network includes a lightweight feature fusion module for intermediate features. and intermediate features Perform feature fusion to obtain fusion features and fusion features .like Figure 7 As shown, the lightweight feature fusion module includes a 1×1 convolution layer, an upsampling layer and a downsampling layer; the lightweight feature fusion module includes an upper branch and a lower branch; the upper branch converts the intermediate features output by the backbone network into After 1×1 convolution layer, the first branch and the second branch are formed; the lower branch takes the intermediate features output by the backbone network After passing through the 1×1 convolution layer, the third and fourth branches are formed; the second branch passes through the upsampling layer and the 1×1 convolution layer and is added to the third branch, and the fusion feature is output. ; The fourth branch is added to the first branch after the downsampling layer and the 1×1 convolution layer to output the fusion feature The head network includes two lightweight detection head modules for fusion features. and fusion features Perform feature decoupling to obtain the confidence, category, and bounding box of the blade tip area. Figure 8 As shown in the figure, the lightweight detection head module includes a depth-separable convolution layer, a 1×1 convolution layer and a two-dimensional convolution layer; the two lightweight detection head modules are composed of three branches, each of which includes a depth-separable convolution layer, a 1×1 convolution layer and a two-dimensional convolution layer; each branch first converts the fusion features output by the neck network into or fusion features After passing through a layer of depth-wise separable convolutional layer, and then two layers of 1×1 convolutional layers, it passes through a layer of two-dimensional convolutional layer, and the three branches output confidence, category and border respectively.

[0020] The training and deployment process of the Mobile-Blade network model is as follows: blade tip images are collected in a real environment and a virtual environment of the software, the collected blade tip images are labeled using the LabelImg annotation tool to obtain corresponding label files, and the labeled information files are converted into text files. The images and text files are respectively stored in two files under the same directory. The text file contains the number, coordinate position, and length and width information of all blade tips; the blade tip images and the corresponding label files are combined into a blade tip image dataset, and the blade tip dataset is randomly divided into a training set, a test set, and a validation set in a ratio of 8:1:1; in the actual experimental process, if there are blade tip images with incorrect positioning, they are added to the blade tip image dataset for data enhancement. The blade tip image dataset of the embodiment of the present invention is as follows: Figure 4 As shown. Use the blade tip image dataset to train the Mobile-Blade network model and obtain the optimal model weight file; create a .yaml file to record the properties of the blade tip image dataset, including the training set, test set, and validation set paths, as well as labeled object category information, and configure model training hyperparameters, including dataset properties, model batches, learning rates, and pre-trained models; use the configured files and parameters to train the Mobile-Blade network, use the regression loss function to calculate the error between the predicted box and the true box, and backpropagate the error to update the network weights to achieve network convergence and obtain the optimal detection model. Export the optimal detection model to the .onnx format, and use the TensorRT inference engine to reconstruct the .onnx file to the .engine inference file format. Load the .engine file into the inference engine, control the graphics card for high-speed inference, detect the blade tip image, and obtain the blade tip detection box position information in the image.

[0021] S4: Locate the blade tip reference point and calculate the blade pitch. The specific steps are as follows: S401: The blade tip image collected in step S2 is detected by the Mobile-Blade network model to obtain all blade tip detection frame information in the image. Each blade tip detection frame information includes the X-axis coordinate and Y-axis coordinate of the upper left corner of the detection frame, the width W and the height H of the detection frame; and it is determined whether the number of blade tip detection frames is equal to ,in, Indicates the number of propeller tips that need to be detected correctly. , if the number of propeller tip detection frames is not equal to Then return to step S2 and re-collect the blade tip image. If the number of blade tip detection frames is equal to Then proceed to the next step.

[0022] S402: Keep the center position of each blade tip detection frame unchanged and expand the area ,in , and set the grayscale value outside the blade tip detection frame area to 0 to obtain an extended mask image; perform Otsu threshold segmentation algorithm on the blade tip detection frame area in the extended mask image to obtain a threshold segmentation image; perform morphological closing operation on the threshold segmentation image to obtain a morphological closing operation image; extract all blade tip contours in the morphological closing operation image, and sort them from large to small according to the blade tip contour area, and output the one with the largest area. Tip profiles, where In an embodiment of the present invention, The value of is 15, and the value of n is 2.

[0023] S403: The blade tip profiles with the minimum and maximum Y-axis coordinates are respectively used as the upper blade tip and lower blade tip profiles; the starting point and the end point of the upper blade tip and the lower blade tip are connected, and the point where the connecting line is three-quarters of the distance from the end point is set as the aerodynamic center; the starting point and the end point of the upper blade tip and the lower blade tip are respectively corrected for errors with the aerodynamic center to obtain the blade tip reference point , blade tip reference point , blade tip reference point , blade tip reference point , blade tip reference point and blade tip reference point The blade tip reference point positioning result in the embodiment of the present invention is as follows: Figure 9 shown.

[0024] The formula for calculating the propeller pitch is: ; in, represents the measured blade tip pitch, is the pixel equivalent result of machine vision calibration, Indicates the blade tip reference point , blade tip reference point or blade tip reference point The Y-axis coordinate of any reference point in Indicates the blade tip reference point , blade tip reference point or blade tip reference point The Y-axis coordinate of any reference point in .

[0025] The performance indicators of the embodiment of the present invention were tested under the hardware environment shown in Table 1 below.

[0026] Table 1 Experimental hardware platform

[0027] equipment Model parameters CPU CPU Intel(R) Core(TM)i7-12700 GPU NVIDIA GeForce GTX 3060 Memory 64GB disk Samsung SSD 990 PRO 2TB

[0028] 1. Model comparison test A self-made propeller tip image dataset was used for model comparison testing, and the average accuracy, precision, recall rate, parameter amount, and number of floating-point operations were verified. Among them, the average accuracy was defined as mAP@50-95, the precision was defined as mAP@50, the recall rate R was defined as the ratio of the correctly predicted targets to the number of correct targets in the entire image, the parameter amount was defined as Paramas, and the number of floating-point operations was defined as GFLOPs. After the test was completed, the test results are shown in Table 2. The results show that the implementation of the present invention can achieve the highest average accuracy and precision when the parameter amount and the number of floating-point operations are small.

[0029] Table 2 Model comparison results

[0030] network Average precision Accuracy Recall Parameter quantity / M Floating point operations / G MobileNetV3-small 0.4418 0.3562 0.5000 1.68 6.60 RT-DETR-L 0.6700 0.3100 0.4600 9.40 21.50 YOLOV5 0.9999 0.9955 1.0000 47.03 118.70 YOLOv8s 0.9999 0.9962 1.0000 11.20 28.40 Embodiments of the present invention 1.0000 0.9970 0.9999 2.50 8.40

[0031] 2. Adaptability test A reflective marker was pasted on a cardboard to simulate the position and shape of the actual paddle tip in an outdoor simulated environment. A location close to the actual experimental scene was selected outdoors. Paddle tip images were collected in two different scenes and at different lighting conditions at different times of the day, morning, noon, and evening. A certain number of images were selected for testing. The generalization test results in the simulated environment are shown in the figure below. Figure 10 As shown, Figure 10 (a1) in the image represents an image taken in the morning under backlight conditions. Figure 10 (a2) in the equation represents Figure 10 Positioning of the blade tip region in (a1), Figure 10 (a3) in the expression Figure 10 The aerodynamic center positioning of the blade tip reference point in (a1); Figure 10 (b1) in the image represents an image taken in the afternoon under backlight conditions. Figure 10 (b2) in the equation represents Figure 10 Positioning of the blade tip region in (b1), Figure 10 (b3) in the equation represents Figure 10 The aerodynamic center positioning of the blade tip reference point in (b1); Figure 10 (c1) in the image represents the image taken in backlight conditions at noon. Figure 10 (c2) in the expression Figure 10 Positioning of the blade tip region in (c1), Figure 10 (c3) in the expression Figure 10 The aerodynamic center positioning of the blade tip reference point in (c1); Figure 10 (d1) in the image represents the image taken in the morning under direct sunlight. Figure 10 (d2) in the expression Figure 10 Positioning of the blade tip area in (d1), Figure 10 (d3) in the expression Figure 10 The aerodynamic center positioning of the blade tip reference point in (d1); Figure 10 (e1) in the figure represents an image taken in the afternoon under direct sunlight. Figure 10 (e2) in the expression Figure 10 Positioning of the blade tip region in (e1), Figure 10 (e3) in the expression Figure 10 The aerodynamic center positioning of the blade tip reference point in (e1); Figure 10 (f1) in the image represents the image taken at noon under direct sunlight conditions. Figure 10 (f2) in the expression Figure 10 Positioning of the blade tip region in (f1), Figure 10 (f3) in the expression Figure 10 The aerodynamic center positioning of the blade tip reference point in (f1) is performed; the simulation experimental results show that the recognition rate reaches 100%.

[0032] 3. Detection accuracy test Among the propeller tip images taken in the simulation experiment, 5 groups were randomly selected, with 100 images in each group. The calibration result was 0.7646 mm / pixel. The average error of the propeller pitch in each group of images was calculated. The calculation results are shown in the figure below. Figure 11 As shown in the figure, the maximum average error is 0.61 mm and the minimum is 0.09 mm, and the fluctuation of the overall curve is small, indicating that the embodiment of the present invention is stable and has high precision.

[0033] 4. Detection time test 200 pitch measurement experiments were conducted in a simulated environment. The average time consumption for each step, including blade tip identification, pitch calculation, and image storage, was calculated and statistically analyzed. The statistical results are shown in Table 3. As can be seen from the table, the average total pitch measurement time per image in the simulated environment was 3.2 ms. Different models were deployed on this hardware platform, and the test results are shown in Table 4. The results show that the embodiment of the present invention has the shortest average processing time for blade tip identification per image, requiring only 1.2 ms.

[0034] Table 3 Statistics of average time consumption for each step of the simulation experiment algorithm

[0035] Processing steps Propeller tip identification Propeller pitch calculation Image storage Total time Average time / ms 1.2 1.0 1.0 3.2 Table 4 Average processing time test results of different models

[0036] Model Average processing time / ms RT-DETR-L 5.2 YOLOV5 8.6 YOLOv8s 3.3 Embodiments of the present invention 1.2

[0037] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for real-time monitoring of coaxial rotor blade pitch in a complex environment based on MB network, characterized in that: The steps include: S1: Install an industrial camera at the same horizontal plane as the intersection center of the coaxial rotor blade tips and perform machine vision calibration; S2: Acquire blade tip image; S3: Build a Mobile-Blade network model, and perform model training and deployment; the Mobile-Blade network model includes a backbone network, a neck network, and a head network; the backbone network includes a lightweight bottleneck module for extracting features from the input image to obtain intermediate features and intermediate features The lightweight bottleneck module includes a 3×3 convolution layer, a batch normalization layer, and a ReLU activation layer; the neck network includes a lightweight feature fusion module for intermediate features. and intermediate features Perform feature fusion to obtain fusion features and fusion features The lightweight feature fusion module includes a 1×1 convolution layer, an upsampling layer, and a downsampling layer; the head network includes two lightweight detection head modules for fusion feature and fusion features Perform feature decoupling to obtain the confidence, category, and border of the paddle tip area. The lightweight detection head module includes a depth-separable convolution layer, a 1×1 convolution layer, and a two-dimensional convolution layer. S4: Locate the blade tip reference point and perform blade pitch calculation.

2. The method for real-time monitoring of coaxial rotor blade pitch in a complex environment based on MB network according to claim 1, characterized in that: The industrial camera is installed at the same horizontal plane as the intersection center of the optical axis and the coaxial rotor blade tip, ensuring that its imaging center is at the same horizontal height as the intersection center point of the coaxial rotor blade tip motion trajectory.

3. The method for real-time monitoring of coaxial rotor blade pitch in a complex environment based on MB network according to claim 2, characterized in that: The specific method of the machine vision calibration is: Use a circular calibration template placed at the vertical plane where the coaxial rotor blade tips meet, set the target parameters, and use an industrial camera to collect k circular calibration template images; use the EDCircle algorithm to process each circular calibration template image to obtain the center coordinates and radius of all circular marking points; connect the centers of the circular marking points at the four corners of each circular calibration template image to form a quadrilateral, and use the line connecting the centers of the circular marking points at the upper right corner and the lower left corner as the diagonal to divide the quadrilateral into two triangles; use the machine vision calibration formula to solve and obtain the pixel equivalent result of the machine vision calibration , the specific expression of the machine vision calibration formula is: ； in, 、 and Represents the three side lengths of the i-th triangle in the j-th calibration template image, ; represents the semi-perimeter of the i-th triangle in the j-th calibration template image; Represents the area of ​​the i-th triangle in the j-th calibration template image; Indicates the center distance of the calibration template, Indicates the number of rows in the calibration template. Indicates the number of columns of the calibration template; Indicates the total number of calibration template images collected; represents the pixel equivalent of the j-th calibration template image, j = 1, 2, ..., k; Indicates the pixel equivalent result of machine vision calibration.

4. The method for real-time monitoring of coaxial rotor blade pitch in a complex environment based on MB network according to claim 3, characterized in that: The specific process of collecting the blade tip image is as follows: stick a reflective marker on the blade tip and illuminate the marker with a strobe light, and then use an industrial camera to collect the blade tip image.

5. The method for real-time monitoring of coaxial rotor blade pitch in a complex environment based on MB network according to claim 4, characterized in that: The training and deployment process of the Mobile-Blade network model is as follows: collecting blade tip images in a real environment and a virtual environment of the software, annotating the collected blade tip images using an annotation tool to obtain corresponding label files, and forming a blade tip image dataset with the blade tip images and the corresponding label files; using the blade tip image dataset to train the Mobile-Blade network model to obtain the optimal model weight file; converting the optimal model weight file into a sequence file in .engine format and deploying the sequence file through the TensorRT platform.

6. The method for real-time monitoring of coaxial rotor blade pitch in a complex environment based on MB network according to claim 5, characterized in that: The specific steps for locating the blade tip reference point are: S401: The blade tip image collected in step S2 is detected by the Mobile-Blade network model to obtain all blade tip detection frame information in the image. Each blade tip detection frame information includes the X-axis coordinate and Y-axis coordinate of the upper left corner of the detection frame, the width W and the height H of the detection frame; and it is determined whether the number of blade tip detection frames is equal to ,in, Indicates the number of propeller tips that need to be detected correctly. , if the number of propeller tip detection frames is not equal to Then return to step S2 and re-collect the blade tip image. If the number of blade tip detection frames is equal to Then proceed to the next step; S402: Keep the center position of each blade tip detection frame unchanged and expand the area ,in , and set the grayscale value outside the blade tip detection frame area to 0 to obtain an extended mask image; perform Otsu threshold segmentation algorithm on the blade tip detection frame area in the extended mask image to obtain a threshold segmentation image; perform morphological closing operation on the threshold segmentation image to obtain a morphological closing operation image; extract all blade tip contours in the morphological closing operation image, and sort them from large to small according to the blade tip contour area, and output the n blade tip contours with the largest area, where ; S403: The blade tip profiles with the minimum and maximum Y-axis coordinates are respectively used as the upper blade tip and lower blade tip profiles; the starting point and the end point of the upper blade tip and the lower blade tip are connected, and the point where the connecting line is three-quarters of the distance from the end point is set as the aerodynamic center; the starting point and the end point of the upper blade tip and the lower blade tip are respectively corrected for errors with the aerodynamic center to obtain the blade tip reference point , blade tip reference point , blade tip reference point , blade tip reference point , blade tip reference point and blade tip reference point .

7. The method for real-time monitoring of coaxial rotor blade pitch in a complex environment based on MB network according to claim 6, characterized in that: The formula for calculating the propeller pitch is: ； in, represents the measured propeller pitch, is the pixel equivalent result of machine vision calibration, Indicates the blade tip reference point , blade tip reference point or blade tip reference point The Y-axis coordinate of any reference point in Indicates the blade tip reference point , blade tip reference point or blade tip reference point The Y-axis coordinate of any reference point in .

Images