A helicopter airborne rotor blade tip displacement measurement method based on point light source detection
By using a point light source-based detection method combined with a Gaussian receptive field and a weighted feature fusion network, the accuracy problem of helicopter rotor tip displacement measurement was solved, achieving low-latency, high-precision tip displacement measurement, which supports blade design and performance evaluation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANCHANG HANGKONG UNIVERSITY
- Filing Date
- 2023-06-19
- Publication Date
- 2026-06-26
AI Technical Summary
Accurate measurement of helicopter rotor tip displacement is difficult to achieve under complex lighting conditions in all weather conditions. Traditional ground measurement equipment cannot obtain the rotor tip displacement in real time during actual flight, and inertial navigation systems will produce large errors after long-term operation.
A point light source-based detection method is adopted, which combines a Gaussian receptive field and a weighted feature fusion network. The point light source at the propeller tip is detected by a deep learning network, and images are acquired in real time using an industrial camera and photoelectric sensor. The propeller tip displacement is calculated by combining an error correction algorithm.
It achieves low-latency, high-precision blade tip displacement measurement with a detection accuracy of 99.85% and an error of less than 10mm. It is suitable for real-time measurement of helicopter airborne rotor blade tip displacement and provides data support for blade design and performance evaluation.
Smart Images
Figure CN116659396B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image data processing, specifically relating to an algorithm for detecting small targets by combining Gaussian receptive field and weighted feature fusion network, and a method for measuring the displacement of helicopter airborne rotor tip using a deep learning network to detect tip point light sources. Background Technology
[0002] Helicopters are a distinctive type of aircraft, typically equipped with one or more horizontally driven propellers that enable vertical takeoff and landing, movement in any vector direction, or maintaining a stable state in the air. In actual flight missions, helicopters engage in unsteady flight states such as acceleration / deceleration, sudden stops, turns, and highly skillful maneuvers, placing extremely high demands on the extreme parameters of the rotor blades. The rotor tip displacement is a crucial indicator for blade design and evaluation. During high-speed rotation, the rotor tip cuts through the edge air, generating a high-Mach number effect and accompanied by significant aerodynamic noise, which is the primary source of helicopter noise. Simultaneously, the resulting vortex effect causes unsteady pressure fluctuations at the blade tip, leading to fluctuations in the vertical displacement of the blade tip. For single-rotor helicopters, the rotor tip displacement reflects the working state of the blades, indicating their fatigue level, and provides data support for reducing helicopter noise. For coaxial twin-rotor helicopters, the distance between the upper and lower rotor tips is a critical factor ensuring safe operation. The distance between the upper and lower rotor tips must be controlled within a set range, and the displacement of each rotor tip must be within a defined envelope.
[0003] The main challenges in measuring helicopter rotor tip displacement are: First, the high speed and strong vibrations of the rotor tip motion make it impossible to embed large tip marking devices, and the measurement equipment requires high processing speeds. Second, the rotor tip area is far from the fuselage, and the pixel size of the area marked by a point light source is extremely limited, making it difficult to detect small targets under complex lighting conditions in all weather conditions. Traditional ground-based measurement equipment cannot acquire the rotor tip displacement in real time during actual flight. While using an inertial navigation system for helicopter rotor tip displacement measurement is simple and low-cost, it suffers from drift over extended periods, leading to significant errors. Therefore, researching real-time measurement of helicopter rotor tip displacement has significant research importance and application value. Summary of the Invention
[0004] To address the shortcomings of the prior art, this invention provides a method for measuring the displacement of helicopter airborne rotor blade tips based on point light source detection, so as to achieve accurate measurement of the displacement of helicopter airborne rotor blade tips.
[0005] The present invention is achieved through the following technical solution.
[0006] A method for measuring the tip displacement of a helicopter rotor based on point light source detection includes the following steps:
[0007] Step (1) Fixing the point light source; Fix the point light source marker at the tip of the rotor blade;
[0008] Step (2) Fixing and calibrating the industrial camera;
[0009] Step (3) Image acquisition of the helicopter rotor tip area; When the point light source on the rotor blade tip enters the central field of view of the industrial camera, the photoelectric sensor will emit a trigger signal, which is converted into a specific frequency pulse signal by the trigger frequency multiplier, thereby triggering the industrial camera to acquire the image of the rotor tip area;
[0010] Step (4) Point source detection based on Gaussian receptive field and weighted feature fusion deep network, the specific steps are as follows:
[0011] Step (4.1) Constructing and augmenting the training sample dataset: First, a large number of images of the paddle tip region under all-weather background are acquired using an industrial camera, and the point light source region is selected by directional bounding using annotation software; Second, the annotated information file is converted into a text file of the dataset pipeline, which includes the coordinate position, length, width and category information of the point light source; Then, the dataset is divided into training set, test set and validation set using a random sampling algorithm, which together with the annotation text file to construct the dataset pipeline; Finally, the input image dataset is augmented, including but not limited to translation, rotation, stitching and sharpening.
[0012] Step (4.2) Constructing a deep learning network model framework; The deep learning network model adopts an end-to-end structure design, and the overall network is built in modules, mainly including a backbone network, a feature fusion network, a Neck, and a task network, a Head. The backbone network is a feature extraction module, mainly composed of 4 convolutional layers, 3 residual networks, and one SPPF layer. The feature fusion network, Neck, includes 4 convolutional layers, 4 residual networks, several concatenation layers (Cat), and several upsampling operation layers (Upsample). The task network, Head, includes a positive and negative sample allocation module for data labels, a loss function calculation module, and a non-maximum suppression module. The backbone network outputs feature maps of 3 receptive fields to the feature fusion network, and then the feature fusion network, Neck, performs scale fusion on the feature maps of the upper and lower receptive fields to obtain high-dimensional semantic information of 3 scales. Finally, the obtained high-dimensional semantic information is input into the task network, Head, for classification and regression of task data.
[0013] Step (4.3) combines the Gaussian receptive field optimization model for the detection of small targets. The Gaussian receptive field optimization steps are as follows: First, the theoretical receptive field is calculated by extracting features using the backbone network constructed in step (4.2); second, the effective receptive field region is extracted as the prior regression bounding box based on the obtained receptive field size; then, both the prior regression bounding box and the manually labeled ground truth bounding box are modeled as two-dimensional Gaussian distribution models as target rectangles. This is beneficial for the edge gradient expansion of small targets and improves the network's extraction of feature information of small targets; then, a new similarity paradigm is used to measure the distance between the Gaussian distribution models of the prior regression bounding box and the manually labeled ground truth bounding box to obtain the score information of all manually labeled ground truth bounding boxes and the assigned samples; finally, a multi-level TopK algorithm is used to allocate the positive and negative samples required for network training.
[0014] Step (4.4) combines channel-weighted feature fusion to optimize the detection of small targets. The weighted feature fusion method will contribute different weights to the output features according to the different resolutions of the input features. The features of small targets are mainly clustered in feature maps with larger resolutions. Therefore, combining channel-weighted feature fusion can improve the attention to small targets. The specific steps of optimizing the model for the detection of small targets by combining channel-weighted feature fusion are as follows: First, design a top-down and bottom-up bidirectional feature fusion network FPN-PANet; second, fuse a simple attention mechanism and use depthwise separable convolution to assign larger weights to feature maps with smaller receptive fields; next, the feature fusion network Neck concatenates the feature maps output by each layer of the backbone network with the output feature maps of the same scale; finally, the fused feature maps are assigned to the task network Head, and the final detection result is output.
[0015] Step (4.5) Model training and deployment;
[0016] Step (5) Point light source position error correction; the linear velocity of helicopter rotor blade tip motion is generally in a subsonic state, characterized by high speed, severe vibration, and large centrifugal force. Since the optical axis of the airborne industrial camera is not collinear with the blade tip motion trajectory, directly calculating the blade tip displacement based on pixel equivalent has a large depth error. The point light source position error correction mainly includes: first, determining the point light source marker point as the reference point, and determining the pixel distance occupied by the vector distance between the blade tip and the reference point under different flapping amounts according to the pinhole imaging principle; then, calculating the functional relationship between the blade tip displacement and the pixel distance under the allowable residual through the fitting function.
[0017] Step (6) Calculate the rotor tip displacement; input the coordinates of the point light source markers in the image calculated by the deep learning network model into the fitting function obtained in step (5) to correct and calculate the actual rotor tip displacement.
[0018] Furthermore, the point light source markers include LED devices, all of which are fixed to the lower surface area of the blade tips.
[0019] Furthermore, the industrial camera is fixed to the helicopter hub, and the angle between the optical axis of the industrial camera and the horizontal plane of the helicopter hub is 45 degrees; then, the selected industrial camera is single-target calibrated to obtain planar pixel equivalent information.
[0020] Furthermore, the calibration method for industrial cameras includes the following steps: First, the industrial camera is connected to a processing computer; second, a calibration target with a rectangular array of circular markers is placed at the point light source marker point, and the target template parameters are set on the processing computer; then, multiple images of the calibration target are acquired using the industrial camera, and the calibration value at each marker point in each image is obtained based on the target template parameters and the proportion of the actual circular marker point's pixel length in the image; the specific formula for calculating the calibration value at the marker point is as follows:
[0021]
[0022] In the formula, PixelEquivalentv is the calibrated pixel equivalent result, N is the number of calibrated image acquisitions, and DA i DP represents the actual distance between the circular markers captured in the i-th image. i The pixel length between the circular markers captured in the i-th image.
[0023] Furthermore, the industrial camera needs to determine its exposure time based on the linear velocity of the blade tip to reduce image retention of the target; at the same time, it needs to ensure effective imaging of the point light source, so the exposure time should be set to no more than 50 microseconds.
[0024] Furthermore, in step (4.3), the theoretical receptive field calculation formula is as follows:
[0025]
[0026] In the formula, TRF n+1 Indicates the receptive field size of the current convolutional layer, TRF n The kernel size is the receptive field size of the previous convolutional layer. n+1 The stride is the kernel size used in the current convolutional layer. i The stride size used for each convolutional layer;
[0027] Therefore, the effective receptive field size of each layer of the network can be obtained as ERF = TRF·γ, where γ is the iteration factor and TRF is the theoretical receptive field of each layer of the network.
[0028] Furthermore, in step (4.3), the calculation formula for modeling the target rectangle as a two-dimensional Gaussian model is as follows:
[0029]
[0030] In the formula, G(x,y) represents the high-dimensional weights after Gaussian modeling, (x,y) are the pixel coordinates, σ is the square of the bounding box, μ is the center coordinates of the bounding box, and Center is the center of the bounding box. obj The coordinates are the center coordinates of the target bounding box.
[0031] Furthermore, in step (4.3), the formula for calculating the new paradigm is:
[0032]
[0033] In the formula, Distance represents the similarity distance between the effective receptive fields of the manually annotated ground truth bounding boxes after Gaussian modeling, and GT k ERF is a manually labeled ground truth bounding box that conforms to a Gaussian distribution. k An effective receptive field box that conforms to a Gaussian distribution.
[0034] Furthermore, the specific steps of model training and deployment in step (4.5) are as follows: First, configure the dataset pipeline, hyperparameters, and pre-trained model for network training; second, supervise network training using the loss function and validation dataset results to obtain the optimal network model; then, rewrite the network model into a .wts format text file and serialize and reconstruct it into a .engine inference file using the TensorRT engine; finally, load the .engine file into the inference engine for high-speed inference.
[0035] The beneficial effects of the present invention are: (1) Unlike the ground measurement device for aviation, the helicopter airborne rotor tip displacement measurement method of the present invention is applied to the helicopter edge measurement device, which has the advantages of low delay, strong anti-interference and no packet loss, and can measure the unsteady helicopter rotor tip displacement data in real time. (2) A deep network structure for detecting small targets was designed, which overcomes the difficulty of detection due to the limited pixel size of point light source imaging. Its detection accuracy for point light sources can reach 99.85%. (3) The trained network model is deployed on the TensorRT engine. The detection time for a single image is only 2ms, realizing continuous detection of point light sources and high-speed measurement of rotor tip displacement. (4) The prior squint error compensation parameter corrects the error of the network detection data. It mainly compensates for the scale information due to the lack of depth information. The maximum error of the corrected rotor tip displacement is less than 10mm. The present invention combines deep learning network and computer vision to realize the measurement of helicopter airborne rotor tip displacement. It has the advantages of low latency, high precision, strong anti-interference and easy deployment. The measured data can provide data support for helicopter blade design and performance evaluation and rotor tip aerodynamic characteristics research, which is conducive to the research and development improvement and flight status monitoring of helicopters. Attached Figure Description
[0036] Figure 1 This is a flowchart illustrating the overall workflow of the helicopter airborne rotor tip displacement measurement method of the present invention.
[0037] Figure 2 This is a schematic diagram of the airborne hardware equipment layout used in the helicopter airborne rotor tip displacement measurement method of the present invention;
[0038] Figure 3 This is a schematic diagram of the computer vision calibration target structure for the helicopter airborne rotor tip displacement measurement method of the present invention.
[0039] Figure 4 This is a deep learning network structure diagram of the helicopter airborne rotor tip displacement measurement method of the present invention.
[0040] Figure 5 This is a Gaussian receptive field modeling diagram of the helicopter airborne rotor tip displacement measurement method of the present invention;
[0041] Figure 6 This is a diagram showing the target detection results of the point light source in the helicopter airborne rotor tip displacement measurement method of the present invention;
[0042] Figure 7 This is a schematic diagram of the prior squint error compensation method for the helicopter airborne rotor tip displacement measurement method of the present invention. Detailed Implementation
[0043] To better understand the above-described objects, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention; however, the invention may be practiced in other ways different from those described herein, and therefore, the invention is not limited to the specific embodiments disclosed below.
[0044] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art described herein. The terms “first,” “second,” and similar terms used in this patent application specification and claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, terms such as “connected” or “linked” are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Terms such as “upper,” “lower,” “left,” and “right” are used only to indicate relative positional relationships, which change accordingly when the absolute position of the described object changes.
[0045] Example:
[0046] This embodiment presents a helicopter airborne rotor tip displacement measurement method based on point light source detection. The overall workflow of this rotor tip displacement measurement method is as follows: Figure 1 As shown, its hardware device layout is as follows: Figure 2 As shown, the hardware equipment required for the measurement method of the present invention includes: a point light source marker 1, a rotor blade 2, an industrial camera 3, a photoelectric sensor 4, a trigger frequency multiplier 5, and a processing computer 6. The industrial camera 3, photoelectric sensor 4, and trigger frequency multiplier 5 are all electrically connected to the processing computer 6, enabling the transmission of collected data to the processing computer 6. The rotor tip displacement measurement method of this embodiment specifically includes the following steps:
[0047] Step (1) Fix the point light source. The fixing method is as follows: Figure 2 As shown, the point light source marker 1 uses an LED device. The LED device consists of a metal casing and a light-emitting diode, and features a wide operating temperature range, stable light intensity, and high stress resistance. This method requires designing a receiving hole with a diameter of approximately 15mm on the lower surface of the rotor blade tip 2. An LED device is fixedly installed in each blade tip receiving hole and sealed with aviation-grade adhesive.
[0048] Step (2) Fixing and calibrating the industrial camera. The device for fixing and calibrating the industrial camera is as follows: Figure 2 and Figure 3As shown, this embodiment uses an industrial camera 3 (model VCXG-13M.I.XT) along with a trigger frequency multiplier 5 and a calibration target to form an industrial camera calibration device. Using the horizontal plane at the helicopter wheel hub as a reference plane, the industrial camera is fixed at a 45-degree angle to the bottom wheel hub of the helicopter to ensure that the central optical axis of the camera is aligned with the point light source marker 1. The industrial camera calibration method includes the following steps: First, the industrial camera 3 is connected to a processing computer 6; second, a calibration target with manually set circular markers arranged in a rectangular array is placed at the point light source marker 1, and the target template parameters are set on the processing computer 6. A schematic diagram of the calibration target with circular markers is shown below. Figure 3 As shown; next, multiple calibration target images are acquired using industrial camera 3. The calibration value at each marker point in each image is obtained based on the target template parameters and the proportion of the actual circular marker point's pixel length in the image. The specific formula for calculating the calibration value at the marker point is:
[0049]
[0050] In the formula, PixelEquivalentv is the calibrated pixel equivalent result, N is the number of calibrated image acquisitions, and DA i DP represents the actual distance between the circular markers captured in the i-th image. i The pixel length between the circular markers captured in the i-th image.
[0051] Step (3) Image acquisition of the helicopter rotor tip area. In this embodiment, the industrial camera needs to determine its exposure time based on the linear velocity of the blade tip to reduce the afterimage phenomenon of the target. At the same time, it is necessary to ensure that the point light source imaging is effective, so the exposure time is set to 50 microseconds. In this embodiment, a GA42Y-775 motor is used to simulate the helicopter main rotor, and a 25K high-hardness engraving plate is used to simulate the rotor blade. The rotor blade is connected to the main rotor through the main rotor shaft, and the rotation signal of the main rotor shaft is obtained by photoelectric sensor 4. When the point light source on the rotor blade tip enters the center field of view of the industrial camera 3, the photoelectric sensor 4 will emit a trigger signal, which is converted into a specific frequency pulse signal by the trigger frequency multiplier 5, thereby triggering the industrial camera 3 to acquire the image of the blade tip area.
[0052] Step (4) Point source detection based on Gaussian receptive field and weighted feature fusion deep network, the specific implementation steps are as follows:
[0053] Step (4.1) Constructing the training sample dataset and its augmentation. First, a large number of images of the paddle tip region under all-weather backgrounds were acquired using an industrial camera, and the point light source regions were oriented and bounded using labelimg annotation software. Second, the annotated information files were converted into text files for the dataset pipeline, conforming to the VOC (Visual Object Classes) dataset format. The text files for the dataset pipeline mainly include the coordinates, dimensions, and category information of the point light sources. Next, a random sampling algorithm was used to divide the dataset into training, testing, and validation sets. The training set accounted for 70%, mainly used to extract the features of point light sources under different environments, calculate the gradient information of the point light sources by the deep learning network, and continuously optimize the parameters of the deep convolutional network model through backpropagation. The testing set accounted for 10%, mainly used to test the detection performance of the network without training images and to test the generalization ability of the model. The validation set accounted for 20%, mainly used to verify the effect of each round of network training during the training process, thereby better supervising the network training. The acquired images and the annotated text files together constitute the dataset pipeline. Finally, data augmentation was performed on the input image dataset, including but not limited to rotation, stitching, and sharpening. Rotation refers to rotating the entire image dataset around the center at a certain angle; stitching refers to combining multiple images into one image and formatting it to a uniform size; and sharpening refers to adjusting the image's HSV color information to adapt to different lighting environments.
[0054] Step (4.2) constructs the deep learning network model framework. The deep learning network model in this embodiment adopts an end-to-end structure design, with the overall network built in modules. For example... Figure 4As shown, the network mainly includes a backbone network (Backbone), a feature fusion network (Neck), and a task network (Head). The backbone network is the feature extraction module, consisting of four convolutional layers, three residual networks, and one SPPF layer. In the figure, "Conv" represents a convolution operation, and "[3,2]" indicates that the operation uses a 3×3 kernel with a stride of 2, and so on. Convolutional operations are mainly used to extract deep semantic information from images and continuously compress and encode the data to adapt to computer computation. "Res_C3" is a residual network based on the CSP architecture, containing three standard convolutional layers and one residual block. The two are concatenated and output through the SiLu activation function, mainly to prevent overfitting and retain richer texture gradient information. "SPPF" is for spatial feature fusion, mainly addressing the multi-scale problem of the target. The feature fusion network Neck consists of four convolutional layers, four residual networks, several concatenation layers (Cat), and an upsampling layer (Upsample). The task network Head includes positive and negative sample assignment for data labels, loss function calculation, and non-maximum suppression. The backbone network outputs feature maps from three receptive fields to the feature fusion network Neck. The Neck performs scale fusion on the feature maps from the upper and lower receptive fields to obtain high-dimensional semantic information at three scales. Finally, this high-dimensional semantic information is input into the task network Head for classification and regression of the task data.
[0055] Step (4.3) combines the Gaussian receptive field optimization with the deep learning network model constructed in step (4.2) to detect small targets. The Gaussian receptive field modeling in this embodiment is as follows: Figure 5 As shown, the specific steps of this method for detecting small targets using a Gaussian receptive field optimization model are as follows: First, abandoning the traditional manual scale grouping, a backbone network is used to extract features to calculate the theoretical receptive field. The theoretical receptive field calculation formula obtained from the standard convolutional layer is as follows:
[0056]
[0057] In the formula, TRF n+1 Indicates the receptive field size of the current convolutional layer, TRF n The kernel size is the receptive field size of the previous convolutional layer. n+1 The stride is the kernel size used in the current convolutional layer. iThis refers to the stride size used for each convolutional layer. From this, the effective receptive field size of each layer of the network, ERF = TRF·γ, can be obtained, where γ is the iteration factor, and TRF is the theoretical receptive field of each layer of the network. In this embodiment, the iteration factor γ is set to 0.5. This is used to dynamically adjust the ERF size. After calculation, the effective receptive field sizes of all P2, P3, P4, and P5 convolutional layers are [10, 18, 34, 66]. Next, the effective receptive field regions are extracted as prior regression bounding boxes based on the obtained receptive field sizes. Then, the prior regression bounding boxes of the extracted effective receptive field regions and the manually labeled ground truth boxes are used as target rectangles and both are modeled as two-dimensional Gaussian distribution models. The formula for modeling the target rectangles as two-dimensional Gaussian models is:
[0058]
[0059] In the formula, G(x,y) represents the high-dimensional weights after Gaussian modeling, (x,y) are the pixel coordinates, σ is the square of the bounding box, μ is the center coordinates of the bounding box, and Center is the center of the bounding box. obj The coordinates of the center of the target box are used, which is beneficial for edge gradient expansion of small targets and improves the network's extraction of feature information of small targets. Then, a new similarity paradigm is used to measure the distance between the prior regression bounding boxes and the manually labeled ground truth boxes using a Gaussian distribution model, obtaining the score information of all manually labeled ground truth boxes and their assigned samples. The calculation formula for the new paradigm is:
[0060]
[0061] In the formula, Distance represents the similarity distance between the effective receptive fields of the manually annotated ground truth bounding boxes after Gaussian modeling, and GT k ERF is a manually labeled ground truth bounding box that conforms to a Gaussian distribution. k To ensure an effective receptive field box conforming to a Gaussian distribution, a multi-level TopK algorithm is used to allocate the positive and negative samples required for network training.
[0062] Step (4.4) combines channel-weighted feature fusion to optimize the detection of small targets. The weighted feature fusion in this embodiment differs from traditional feature pyramid fusion. The method in this embodiment will assign different weights to the output features based on the different resolutions of the input features. The weighted feature fusion network structure is as follows: Figure 4The feature fusion network Neck module is shown in the diagram. Features of small targets are mainly clustered in feature maps with higher resolution. Therefore, combining channel-weighted feature fusion can improve the focus on small targets. The specific steps of this method, combining channel-weighted feature fusion to optimize the model for detecting small targets, are as follows: First, a top-down, bottom-up bidirectional feature fusion network FPN-PANet is designed to enhance the extraction of detailed information about small targets. Second, a simple attention mechanism is fused, using depthwise separable convolutions to assign larger weights to feature maps with smaller receptive fields. This process is similar to softmax classification, adding batch normalization and ReLU activation functions after each standard convolution to assign corresponding weights to each layer. Next, the feature fusion network Neck concatenates the P2, P3, and P4 feature layers extracted from the backbone network with the fused output feature map. Finally, the resulting feature map is assigned to the task network Head, outputting the final detection result.
[0063] Step (4.5) Model Training and Deployment. The specific steps for model training and deployment in this embodiment are as follows: First, configure the dataset pipeline, hyperparameters, and pre-trained model for network training. The dataset used is the self-made paddle tip region image dataset obtained above. Training uses the SGD optimizer, with the initial learning rate set to 0.01 and the maximum anchor regression ratio set to 4, among other hyperparameters. Second, supervise network training using the loss function and validation dataset results to obtain the optimal network model. Next, rewrite the network model into a .wts format text file and serialize and reconstruct it into a .engine inference file using the TensorRT engine. Finally, load the .engine file into the inference engine for high-speed inference, and the output result is as follows. Figure 6 As shown, the results demonstrate that the trained network model exhibits strong robustness in point light source detection under different illumination intensities. The relevant test metrics of this invention were tested under the training parameters shown in Table 1, and the results are as follows.
[0064] Table 1 Model Training and Accelerated Computation Parameters
[0065]
[0066] 1. Detection accuracy test.
[0067] The deep learning network constructed in step (4) was used to train the collected image dataset, which contained paddle tip images with different backgrounds and exposures. 70% of the images were divided into the training set, 20% into the validation set, and 10% into the test set. The loss value was defined as the distance between the manually labeled ground truth bounding boxes and the network prediction results. The recall rate was defined as the proportion of the predicted target to all targets in the entire image. The precision was defined as map@50, and the average precision was defined as map@50-95. The number of training epochs was set to 500. After testing, the loss value, recall rate, precision, and average precision of the paddle tip point light source detection were obtained in Table 2.
[0068] Table 2 Detection accuracy test results
[0069]
[0070]
[0071] 2. Detection time test.
[0072] The test method performed in this embodiment starts timing from acquiring an image frame from the industrial camera, and ends timing after the network forward propagates to obtain the processing result. The difference is the processing time for a single image. The average processing time is obtained by calculating the processing time for all image frames, and the test results are shown in Table 3. The industrial camera used in this embodiment has a frame rate of 91 frames per second, and the average processing frame rate using the TensorRT engine is 125 frames per second, which meets the real-time processing requirements.
[0073] Table 3. Detection time test results of the network model deployed on different platforms.
[0074]
[0075] Step (5) Point light source position error correction; the linear velocity of helicopter rotor blade tip motion is generally in a subsonic state, characterized by high speed, severe vibration, and large centrifugal force. Since the optical axis of the airborne industrial camera is not collinear with the blade tip motion trajectory, directly calculating the blade tip displacement based on pixel equivalents has a large depth error. The error correction principle in this embodiment is as follows: Figure 7 As shown, the point light source position error correction mainly includes: based on Figure 7 The process begins by determining the blade tip reference point. Based on the pinhole imaging principle, the pixel distance L corresponding to the vector distance between the rotor blade tip T and the reference point O under different blade tip flapping amounts H is then determined. Finally, the functional relationship between the blade tip displacement D and the pixel distance L under the allowable residual is calculated using a fitting function. This fitting uses second-order parameters with a residual norm less than 0.01.
[0076] Step (6) calculates the rotor tip displacement. The Euclidean distance between the point light source marker and the rotor tip reference point in the image is calculated by the deep learning network model constructed by this method and the product of the calculated distance is the pixel distance L. The pixel distance is input into the function obtained in step (5) and corrected to the rotor tip displacement D. The measurement results are shown in Table 4.
[0077] Table 4. Results of Strabismus Error Compensation (Unit: mm)
[0078]
[0079] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for measuring the tip displacement of a helicopter rotor based on point light source detection, characterized in that, Includes the following steps: Step (1) Fixing the point light source; Fix the point light source marker at the tip of the rotor blade; Step (2) Fixing and calibrating the industrial camera; Step (3) Acquire images of the helicopter rotor tip area using an industrial camera; Step (4) Point source detection based on Gaussian receptive field and weighted feature fusion deep network, the specific steps are as follows: Step (4.1) Constructing and augmenting the training sample dataset: First, a large number of images of the paddle tip region under an all-weather sky background are acquired using an industrial camera, and the point light source regions are selected using annotation software; Second, the annotated information files are converted into text files for the dataset pipeline, which include the coordinates, dimensions, and category information of the point light sources; Then, a random sampling algorithm is used to divide the dataset into training, testing, and validation sets, which, together with the annotated text files, form the dataset pipeline; Finally, data augmentation is performed on the input image dataset, including but not limited to translation, rotation, stitching, and sharpening; Step (4.2) Constructing a deep learning network model framework; The deep learning network model adopts an end-to-end structure design, and the overall network is built in modules, mainly including a backbone network, a feature fusion network, a Neck, and a task network, a Head. The backbone network is a feature extraction module, mainly composed of 4 convolutional layers, 3 residual networks, and one SPPF layer. The feature fusion network, Neck, includes 4 convolutional layers, 4 residual networks, several concatenation layers (Cat), and several upsampling operation layers (Upsample). The task network, Head, is used for classification and regression of task data. The backbone network outputs feature maps of 3 receptive fields to the feature fusion network, Neck, which then performs scale fusion on the feature maps of the upper and lower receptive fields to obtain high-dimensional semantic information of 3 scales. Finally, the obtained high-dimensional semantic information is input into the task network, Head, for classification and regression of task data. Step (4.3) combines the Gaussian receptive field optimization model for the detection of small targets. The Gaussian receptive field optimization steps are as follows: First, the theoretical receptive field is calculated by extracting features using the backbone network constructed in step (4.2); second, the effective receptive field region is extracted as the prior regression bounding box based on the obtained receptive field size; then, both the prior regression bounding box and the manually labeled ground truth bounding box are modeled as target rectangles as two-dimensional Gaussian distribution models to improve the network's extraction of small target feature information; then, a new similarity paradigm is used to measure the distance between the two Gaussian distribution models to obtain the score information of all manually labeled ground truth bounding boxes and the assigned samples; finally, a multi-level TopK algorithm is used to allocate the positive and negative samples required for network training. Step (4.4) combines channel-weighted feature fusion to optimize the model for detecting small targets; the weighted feature fusion method will have different weight contributions to the output features according to the different resolutions of the input features; the specific steps of optimizing the model for detecting small targets by combining channel-weighted feature fusion are as follows: First, design a top-down, bottom-up bidirectional feature fusion network FPN-PANet; second, fuse a simple attention mechanism and use depthwise separable convolution to assign larger weights to feature maps with smaller receptive fields; next, the feature fusion network Neck concatenates the feature maps of each layer output by the backbone network Backbone with the output feature maps of the same scale; finally, the fused feature maps are assigned to the task network Head, and the final detection result is output; Step (4.5) Model training and deployment; Step (5) Correction of point light source position error; the specific steps are as follows: first, determine the point light source marker as the reference point, and determine the pixel distance occupied by the vector distance between the blade tip and the reference point under different flapping amounts according to the pinhole imaging principle; then calculate the functional relationship between the blade tip displacement and the pixel distance under the allowable residual through the fitting function. Step (6) Calculate the rotor tip displacement; input the coordinates of the point light source markers in the image calculated by the deep learning network model into the fitting function obtained in step (5) to correct and calculate the actual rotor tip displacement.
2. The method for measuring helicopter airborne rotor tip displacement based on point light source detection according to claim 1, characterized in that, The point light source markers include LED devices, all of which are fixed to the lower surface area of the tips of all blades.
3. The method for measuring helicopter airborne rotor tip displacement based on point light source detection according to claim 1, characterized in that, The industrial camera is fixed to the helicopter hub, and the angle between the optical axis of the industrial camera and the horizontal plane of the helicopter hub is 45 degrees. Then, the selected industrial camera is single-target calibrated to obtain planar pixel equivalent information.
4. The method for measuring helicopter airborne rotor tip displacement based on point light source detection according to claim 3, characterized in that, The method for single-target calibration using an industrial camera includes the following steps: First, connect the industrial camera to a processing computer; second, place a calibration target with a rectangular array of circular markers at the LED marker points, and set the target template parameters on the processing computer; then, use the industrial camera to acquire multiple images of the calibration target, and obtain the calibration value at each marker point in each image based on the target template parameters and the proportion of the actual circular marker points to the pixel length in the image; the specific formula for calculating the calibration value at the marker point is as follows: In the formula, PixelEquivalentv is the calibrated pixel equivalent result, N is the number of calibrated image acquisitions, and DA i DP represents the actual distance between the circular markers captured in the i-th image. i The pixel length between the circular markers captured in the i-th image.
5. The method for measuring helicopter airborne rotor tip displacement based on point light source detection according to claim 1, characterized in that, The exposure time for industrial cameras should be set to no more than 50 microseconds.
6. The method for measuring helicopter airborne rotor tip displacement based on point light source detection according to claim 1, characterized in that, In step (4.3), the theoretical receptive field calculation formula is: In the formula, TRF n+1 Indicates the receptive field size of the current convolutional layer, TRF n The kernel size is the receptive field size of the previous convolutional layer. n+1 The stride is the kernel size used in the current convolutional layer. i The stride size used for each convolutional layer; The effective receptive field size of each layer of the network can be obtained as REF = TRF·γ, where γ is the iteration factor and TRF is the theoretical receptive field of each layer of the network.
7. The method for measuring helicopter airborne rotor tip displacement based on point light source detection according to claim 1, characterized in that, In step (4.3), the calculation formula for modeling the target rectangle as a two-dimensional Gaussian model is as follows: In the formula, G(x,y) represents the high-dimensional weights after Gaussian modeling, (x,y) are the pixel coordinates, σ is the square of the bounding box, μ is the center coordinates of the bounding box, and Center is the center of the bounding box. obj The coordinates are the center coordinates of the target bounding box.
8. The method for measuring helicopter airborne rotor tip displacement based on point light source detection according to claim 1, characterized in that, In step (4.3), the formula for calculating the new paradigm is: In the formula, Distance represents the similarity distance between the effective receptive fields of the manually annotated ground truth bounding boxes after Gaussian modeling, and GT k ERF is a manually labeled ground truth bounding box that conforms to a Gaussian distribution. k An effective receptive field box that conforms to a Gaussian distribution.
9. The method for measuring helicopter airborne rotor tip displacement based on point light source detection according to claim 1, characterized in that, The specific steps for model training and deployment in step (4.5) are as follows: First, configure the dataset pipeline, hyperparameters, and pre-trained model for network training; second, supervise network training using the loss function and validation dataset results to obtain the optimal network model; then, rewrite the network model into a .wts format text file, and serialize and reconstruct the .wts format text file into a .engine inference file using the TensorRT engine; finally, load the .engine file into the inference engine for high-speed inference.