Fan blade tip recognition algorithm of airborne edge computing platform of unmanned aerial vehicle

Through the drone-on-air edge computing platform combining deep learning models and video tracking algorithms, the problems of low monitoring efficiency and insufficient recognition accuracy of traditional fan blades are solved, and high-precision real-time identification and tracking of fan blade tips are realized to adapt to different environments and Blade type, improving monitoring efficiency and accuracy.

CN119007053AActive Publication Date: 2024-11-22NANTONG HUOYAN INTELLIGENT TECH CO LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202411489192.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-24
Publication Date
2024-11-22
Estimated Expiration
2044-10-24

AI Technical Summary

Technical Problem

Traditional fan blade monitoring methods are inefficient and costly, and it is difficult to accurately identify the tip position in complex environments. The existing automated monitoring technology is easily limited by computing resources and processing speed when processing large amounts of data, making it difficult to meet real-time monitoring and rapid response. need.

Method used

Adopting a drone-on-board edge computing platform, combining high-precision sensors and deep learning models, the video tracking algorithm uses gradient information and pixel point weight matrix to achieve high-precision recognition and real-time tracking of blade tips, and the automated control system ensures that the drone is in accordance with Pre-order strategies to approach and identify leaf tips.

Benefits of technology

It realizes high-precision and real-time identification of fan blade tips, shortens response time, improves monitoring efficiency and accuracy, provides reliable technical support for fan maintenance, and has strong adaptability to cope with different environmental conditions and blades. type.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007053A_ABST
    Figure CN119007053A_ABST
Patent Text Reader

Abstract

The invention discloses a fan blade tip recognition algorithm of an unmanned aerial vehicle airborne edge computing platform, and relates to the technical field of wind power equipment maintenance, and the algorithm comprises the following steps: 1, installing a flexible copper net formed by four supports at the top of an unmanned aerial vehicle, leading out a wire at the connection part of the supports and the unmanned aerial vehicle, and in the flight process of the unmanned aerial vehicle, carrying out the recognition of a blade tip of a fan blade; by integrating a high-precision sensor and a deep learning model, high-precision recognition of the blade tip of the fan blade is realized, the multi-layer structure of the deep learning model ensures that the model can process complex and changeable blade images, the recognition accuracy is improved, and meanwhile, the recognition accuracy is improved. According to the video tracking algorithm, gradient information and a pixel point weight matrix are utilized, high-dynamic and low-delay tracking of the blade tip is achieved, the real-time performance is ensured, the recognition precision is improved, the response time is shortened, and reliable technical support is provided for monitoring and maintenance of the fan blade.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of wind power equipment maintenance, and specifically to an algorithm for identifying the tip of a wind turbine blade by an edge computing platform onboard an unmanned aerial vehicle. Background Art

[0002] In the field of wind power generation, wind turbine blades are key components, and their health directly affects the operating efficiency and safety of the wind turbine. However, traditional wind turbine blade monitoring methods often have many shortcomings.

[0003] On the one hand, traditional methods mostly rely on manual inspections, which are inefficient and costly, and it is difficult to detect minor damage or abnormalities on the blades in a timely manner. On the other hand, although the existing automated monitoring technology can reduce manual intervention to a certain extent, it still has obvious shortcomings in recognition accuracy and real-time performance. Especially under complex and changeable environmental conditions, such as insufficient light, high wind speed or when the blades are rotating, traditional monitoring technology often finds it difficult to accurately capture the position information of the blade tip, resulting in deviations in the monitoring results. In addition, when processing large amounts of data, the existing monitoring system is also easily limited by computing resources and processing speed, making it difficult to meet the needs of real-time monitoring and rapid response.

[0004] Therefore, developing an innovative technology that can identify the tips of wind turbine blades with high precision and in real time has become an urgent need to improve the efficiency and accuracy of wind turbine monitoring. Summary of the invention

[0005] The purpose of the present invention is to make up for the shortcomings of the prior art and provide an algorithm for identifying the tips of wind turbine blades by an edge computing platform on a drone. It can achieve high-precision identification of the tips of wind turbine blades by integrating high-precision sensors and deep learning models. The multi-layer structure of the deep learning model ensures that the model can handle complex and changeable blade images, thereby improving the accuracy of recognition. At the same time, the video tracking algorithm utilizes gradient information and pixel weight matrix to achieve high-dynamic and low-latency tracking of the blade tip, ensuring real-time performance. It not only improves the recognition accuracy, but also shortens the response time, providing reliable technical support for the monitoring and maintenance of wind turbine blades.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: an algorithm for identifying the tip of a wind turbine blade by an edge computing platform on a drone, the algorithm comprising the following steps: Step 1: Install a flexible copper mesh consisting of four brackets on the top of the drone, and lead out a wire from the connection between the bracket and the drone. During the flight of the drone, it is suspended to the ground by a tethered method and connected to one end of the direct resistance meter. The other end of the direct resistance meter is connected to the tower base ground wire. At the same time, a measurement system is installed on the drone, and a corresponding automatic control system and algorithm are developed. The measurement system includes a sensor module and a data processing module. The sensor module is responsible for collecting data, and the data processing module processes the collected data. The two work together to ensure that the system can accurately obtain the required information for subsequent algorithm calculations. The automatic control system is used to control the flight attitude and movement of the drone, so that it can approach and identify the blade tip according to a predetermined strategy; Step 2: Lock the blade at a Y-shaped angle, and fly the drone to a certain distance below the blade tip to ensure that the blade tip is in the field of view of the guide camera. The deep learning model is trained based on the neural network structure. The number of neurons in the input layer is set to , whose value is determined by the dimension of the collected wind turbine blade image data set. Suppose the image data set is ,in is the number of images, for each image , whose pixel coordinates are , , is the number of pixels, , the hidden layer adopts a multi-layer structure, and the number of hidden layers is , No. The number of neurons in the hidden layer is , the formula for determining it is ,in is the adjustment coefficient, with a value range of [0.5, 1.5]. The optimal value under different leaf types and environments is determined through experiments. The number of neurons in the output layer is , corresponding to the recognition result of the leaf tip, define the loss function ,in is the predicted value, is the true value, is a regularization parameter with a value range of [0.001, 0.1], which is used to prevent overfitting. For the Tier The weights of neurons are adjusted through the back propagation algorithm to make the loss function Minimize, thus achieving automatic identification of the blade tip; Step 3: Based on the video tracking algorithm, the position coordinates of the leaf tip in the image are defined as , the image center coordinates are , let the gray value matrix of the image be , whose size is , by calculating the gradient matrix of the image , whose elements are and , use the gradient information to determine the initial position of the leaf tip, and then define a weight matrix based on the gray value distribution characteristics of the pixels around the leaf tip , whose elements It is determined by the distance between the pixel points around the leaf tip and the leaf tip and the gray value difference. Let the distance be , the gray value difference is ,but ,in is the weight adjustment coefficient, the value range is [0.1, 1], and the weighted position coordinates are calculated , in order to achieve high-dynamic, low-latency blade tip tracking and obtain the position of the blade tip in the image in real time; Step 4: Combine the tip height with the high-precision RTK information of the UAV and set the tip height to , the horizontal coordinates in the drone RTK information are , define a space transformation matrix , which is constructed based on the comprehensive consideration of the earth curvature, atmospheric refraction and the UAV's own attitude factors. The earth curvature parameter is , the atmospheric refraction coefficient is , the attitude angle of the drone is ,but Elements It is derived through complex geometric and physical models, namely , through the space transformation formula ,in The coordinates of the blade tip in the initial reference system are used to calculate the spatial three-dimensional deviation of the blade tip relative to the drone, thereby obtaining the horizontal control amount of the drone. The horizontal position of the drone is controlled through the interface of the drone SDK to make the blade tip as close to the center of the image as possible. At the same time, the drone is controlled to rise in height, so as to achieve approach and contact with the blade tip through visual guidance.

[0007] Furthermore, the installation angle of the flexible copper mesh installation structure is and location After precise design, the flexible copper mesh composed of four brackets can adapt to different flight postures and environmental interference while ensuring the stability of the connection with the drone. From the perspective of structural mechanics, the length of the bracket and diameter To satisfy a specific proportional relationship, the proportional coefficients are , then , , , This proportional relationship is obtained through a large number of wind tunnel experiments and simulation analysis, in which different wind speeds are considered. ,wind direction And the flight speed of the drone and posture angle The influence of factors on the stability of the copper mesh, at the same time, the conductivity of the copper mesh material and flexibility To meet specific requirements, the thresholds are set as and , then and This material property can ensure good conductivity and the ability to adapt to the shape of the blade surface when in contact with the blade tip.

[0008] Furthermore, the automation control system adopts a hierarchical control architecture, including the bottom hardware driver layer, the middle control algorithm layer and the upper task scheduling layer. The hardware driver layer is responsible for communicating and controlling the hardware devices of the drone, including motor drive and sensor data acquisition. The control algorithm layer adopts a hybrid control strategy, combining proportional integral differential (PID) control and fuzzy logic control. For the horizontal position control of the drone, the desired position is assumed to be , the current location is , the error is , the parameters of PID control are proportional coefficient , integral coefficient and the differential coefficient , and the determination method is as follows: Assume that the dynamic characteristic parameters of the UAV flight environment are , including wind speed ,wind direction and air density , ,in is a coefficient determined by experiments, with a value range of [0.1, 10]. At the same time, fuzzy logic control is used to deal with uncertainty and nonlinear problems. By defining fuzzy sets and fuzzy rules, the parameters of PID control are dynamically adjusted. The upper task scheduling layer reasonably allocates system resources according to the priority and time requirements of the task to ensure efficient operation of the system.

[0009] Furthermore, the data collected by the deep learning model training data include image data of different models of wind turbine blades under different lighting conditions, different angles and different operating states. For the lighting conditions, there are three conditions: strong light, weak light and natural light. A large amount of image data is collected respectively, and the images are normalized. The normalization function is set as ,in is the original image data, and They are the minimum and maximum values ​​of the data respectively. For the blade angle, it covers all angles from horizontal to vertical. The actual angle of the blade is obtained through precise angle measurement equipment and marked in the image data. For the running state, it includes two situations: stationary and rotating. In the rotating state, different speeds are collected At the same time, in order to improve the generalization ability of the model, the collected data was expanded and processed, and data enhancement technology was used, including rotation, flipping and scaling operations. The probability of each operation Through experiments, it was determined that the value range is [0.1, 1]. This targeted data collection and processing method can improve the accuracy and generalization ability of deep learning models.

[0010] Furthermore, in the implementation process of the video tracking algorithm, a multi-scale feature extraction method is adopted. Assume that the original scale of the image is , through downsampling and upsampling operations, we get image features of different scales, which are For each scale of image features, different feature extractors, namely different layer structures of convolutional neural network (CNN), are used. At the same time, in the tracking process, an adaptive window adjustment strategy is adopted, and the initial tracking window size is set to , according to the movement speed of the leaf tip in the image and direction , dynamically adjust the size and position of the tracking window, and set the adjustment coefficient of the tracking window to , then the window adjustment formula is In addition, in order to improve the tracking accuracy, the color and texture features of the pixels are considered when calculating the weights of the pixels in the window. The color value of the pixel is set to , the texture value is , define the weight function as ,in is the weight adjustment coefficient, It is the reference color value and texture value, determined through experiments.

[0011] Furthermore, the spatial solution formula is reasonable. When deriving the spatial solution formula, the influence of the earth's curvature and atmospheric refraction factors are taken into account. Assuming the earth's radius is , the atmospheric refraction coefficient is , the horizontal distance of the blade tip relative to the drone is , the height difference is , define a space transformation matrix , which is constructed based on the comprehensive consideration of the earth curvature, atmospheric refraction and the UAV's own attitude factors. The earth curvature parameter is , the atmospheric refraction coefficient is , the attitude angle of the drone is ,but Elements It is derived through complex geometric and physical models, namely , through the space transformation formula ,in is the coordinate of the blade tip in the initial reference system. At the same time, in order to improve the accuracy of the solution results, in practical applications, the spatial transformation matrix is ​​obtained by real-time acquisition of atmospheric refraction coefficient and earth curvature related data. Dynamic adjustment is performed based on an adaptive algorithm. The adaptive coefficient is set as When the atmospheric refractive index or earth curvature parameter changes exceeding the threshold value, When, yes Elements Update, the update formula is ,in is the change in element calculated based on the new data.

[0012] Furthermore, when the drone SDK interface is used to control the horizontal position and height of the drone, the parameters of the interface are finely set and adjusted. The horizontal control parameters of the interface are set as , the vertical control parameter is Through experiments and simulation analysis, the parameter value range under different control effects is determined. For horizontal position control, when the deviation between the expected position and the current position is within a certain range, the deviation range is set to ,but The value of satisfies ,in and are the lower and upper limits determined by the experiment. For altitude control, when the deviation between the desired altitude and the current altitude is within a certain range, the deviation range is ,but The value of satisfies ,in and are the lower and upper limits determined by the experiment. At the same time, in order to improve the accuracy of control, the flight characteristics of the UAV and the accuracy of the sensor are considered during the control process, and the interface parameters are dynamically adjusted. The adjustment method is based on an adaptive algorithm. The adaptive coefficient is set to When the flight characteristics of the drone or the sensor accuracy changes beyond the threshold, When, yes and Update, the update formula is ,in and is the parameter change calculated according to the new situation.

[0013] Furthermore, the measurement system includes a sensor module and a data processing module. The sensor module uses a high-precision camera and a resistance measurement sensor. The resolution of the camera is ,in and are the number of pixels in the horizontal and vertical directions respectively. By improving the resolution of the camera, a clearer image of the leaf tip can be obtained, which is beneficial for the identification and measurement of the leaf tip. The measurement accuracy of the resistance measurement sensor is By improving the measurement accuracy of the sensor, the resistance value of the blade tip-tower base can be measured more accurately. The data processing module uses a digital signal processing algorithm to filter, amplify and digitize the data collected by the sensor. Suppose the filtering algorithm is , the magnification is , the digital processing function is , then the processed data ,At the same time, in order to improve the accuracy of the ,measurement system, the sensor module and the data processing module were ,coordinately optimized, and the optimal parameter matching relationship between ,the sensor module and the data processing module was determined through ,experiments, including the relationship between the camera resolution and ,filtering algorithm parameters, and the relationship between the resistance ,measurement sensor accuracy and the magnification.

[0014] Furthermore, the blade tip recognition and control process is synergistic. In the whole process, the blade tip recognition, tracking and UAV control are interrelated and coordinated. When the blade tip recognition algorithm detects the position change of the blade tip, it promptly transmits the information to the tracking algorithm and the UAV control algorithm. The tracking algorithm adjusts the tracking strategy according to the position change of the blade tip. The UAV control algorithm adjusts the speed and height of the UAV according to the tracking results and the spatial solution results. At the same time, the control state of the UAV will also be fed back to the blade tip recognition and tracking algorithm so as to adjust the algorithm parameters and strategies in time. This synergy is achieved through a communication protocol. Suppose the communication protocol is It includes provisions on message format, transmission rate and error handling. That is, the message format specifies the encoding method of blade tip position information and UAV control instructions. The transmission rate is determined according to the flight speed of the UAV and the mission requirements. Error handling specifies how to resend and correct errors when information is transmitted incorrectly. This synergy can ensure the efficient operation and accuracy of the entire system.

[0015] Furthermore, the algorithm as a whole is robust, and can maintain stable performance and accuracy when facing different environmental conditions, blade types and UAV performance differences. For environmental conditions, including different wind speeds, wind directions, light and temperatures, by introducing an adaptive mechanism into the algorithm, the algorithm's parameters and strategies can be automatically adjusted according to environmental changes. For blade types, wind turbine blades of different models and designs are covered, and the shapes and structures of different blades can be adapted by extracting and analyzing blade features. For UAV performance differences, the flight characteristics and sensor accuracy differences of UAVs of different brands and models are taken into account. By calibrating and adjusting UAV parameters, the effective operation of the algorithm on different UAVs can be guaranteed.

[0016] Compared with the existing technology, the recognition algorithm of the wind turbine blade tip by the UAV-mounted edge computing platform has the following beneficial effects: 1. The present invention realizes high-precision and real-time recognition and tracking of the blade tips of wind turbine blades through the combination of deep learning models and video tracking algorithms. The deep learning model is based on a neural network structure and is trained with a large amount of image data. It can accurately identify blade tips under different lighting conditions, angles and operating states. The video tracking algorithm uses multi-scale feature extraction and adaptive window adjustment strategies to track the position of the blade tip in the image in real time. Even when the blade tip moves at high speed or the lighting conditions change, the tracking error can be kept small. This high-precision and real-time recognition and tracking capability provides reliable data support for subsequent UAV control, ensuring that the UAV can accurately and stably approach and contact the blade tip.

[0017] 2. The present invention has strong adaptability and can cope with different environmental conditions (such as wind speed, wind direction, light and temperature) and blade types (wind turbine blades of different models and designs). By introducing an adaptive mechanism, the algorithm can automatically adjust parameters and strategies according to environmental changes to ensure stable performance and accuracy under different conditions. At the same time, the algorithm also takes into account the performance differences of drones. By calibrating and adjusting the parameters of drones, the algorithm is guaranteed to run effectively on drones of different brands and models. This powerful adaptive ability makes the algorithm have broad application prospects and practical value in practical applications.

[0018] Other advantages, objectives and features of the present invention will be set forth in part in the following description and, in part, will be apparent to those skilled in the art based on an examination of the following or may be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 The figure is a flow chart of an algorithm for identifying the tip of a wind turbine blade using an edge computing platform mounted on a drone. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Example 1: Identification of blade tips of a specific type of wind turbine in a strong light environment In an open area with plenty of sunshine, the blade tip of a wind turbine model XX is identified. The wind speed on site is , wind direction is The flight speed of the drone is , the attitude angle is .

[0023] Algorithm parameter settings Deep Learning Models: The collected image data set Contains 1000 images , the number of pixels in each image is , then the number of neurons in the input layer ; Number of hidden layers , adjustment coefficient , for the first hidden layer , the number of neurons , the second hidden layer , , the third hidden layer , , the regularization parameter .

[0024] Video Tracking Algorithm: Image gray value matrix Size , weight adjustment coefficient , the initial size of the tracking window is .

[0025] Spatial solution formula: Earth curvature parameters , atmospheric refraction coefficient , the drone attitude angle is , then the space transformation matrix Elements .

[0026] Process and monitoring: First, the drone flies according to the predetermined posture, the flexible copper mesh remains stable, and the sensor module collects data. The camera resolution is , the resistance measurement sensor accuracy is , the deep learning model processes the collected images and continuously adjusts the weights through the back propagation algorithm. After 1000 iterations, the loss function The initial value of 100 was reduced to 1.5, which achieved the preliminary recognition of the blade tip. The video tracking algorithm accurately tracked the blade tip according to the image gradient and pixel weight. During the movement of the blade tip, the window size was dynamically adjusted according to the blade tip speed and direction. The tracking error was always kept within 5 pixels. The spatial solution formula combined the blade tip height and the UAV position information to accurately solve the spatial 3D deviation of the blade tip relative to the UAV, control the horizontal position and height of the UAV, and control the deviation between the blade tip and the image center within 5 pixels. Pixel, height deviation is controlled within Finally, the drone successfully approached the blade tip, the metal mesh came into contact with the blade tip, and the direct resistance meter accurately measured the blade tip-tower base resistance value. .

[0027] Example 2: Identification of blade tips of different types of wind turbines in a weak light environment On a dark evening, the blade tips of two types of wind turbines, model YY and ZZ, were identified. The wind speed at the site was , wind direction is °, , the attitude angle is .

[0028] Algorithm parameter settings Deep Learning Models: The collected image data set Contains 1500 images , the number of pixels in each image is , then the number of neurons in the input layer ; Number of hidden layers , adjustment coefficient , for the first hidden layer ( ), the number of neurons , the second hidden layer , , the third hidden layer , , the 4th hidden layer , , the regularization parameter .

[0029] Video tracking algorithm: Image gray value matrix Size , weight adjustment coefficient , the initial size of the tracking window is .

[0030] Earth curvature parameters , atmospheric refraction coefficient , the attitude angle of the drone is , then the space transformation matrix Elements .

[0031] Process and monitoring: The drone flies in a weak light environment, the flexible copper mesh has good stability, the sensor module collects data, and the camera resolution is , the resistance measurement sensor accuracy is After 1500 iterations of the deep learning model, the loss function The initial value was reduced from 120 to 2.5, achieving preliminary recognition of the leaf tip. The video tracking algorithm accurately tracked the leaf tip by adjusting the weight in weak light, with the tracking error within 8 pixels. The spatial solution formula accurately solved the deviation, and controlled the drone to keep the deviation between the leaf tip and the image center within 1.5. Pixels, height deviation is controlled within For two different types of blades, the blade tip-tower base resistance values ​​can be accurately measured, which are .

[0032] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. An algorithm for identifying the tip of a wind turbine blade using an edge computing platform on a drone, characterized in that: The algorithm consists of the following steps: Step 1: Install a flexible copper mesh consisting of four brackets on the top of the drone, and lead out a wire from the connection between the bracket and the drone. During the flight of the drone, it is suspended to the ground by a tethered method and connected to one end of the direct resistance meter. The other end of the direct resistance meter is connected to the tower base ground wire. At the same time, a measurement system is installed on the drone, and a corresponding automatic control system and algorithm are developed. The measurement system includes a sensor module and a data processing module. The sensor module is responsible for collecting data, and the data processing module processes the collected data. The two work together to ensure that the system can accurately obtain the required information for subsequent algorithm calculations. The automatic control system is used to control the flight attitude and movement of the drone, so that it can approach and identify the blade tip according to a predetermined strategy; Step 2: Lock the blade at a Y-shaped angle, and fly the drone to a certain distance below the blade tip to ensure that the blade tip is in the field of view of the guide camera. The deep learning model is trained based on the neural network structure. The number of neurons in the input layer is set to , whose value is determined by the dimension of the collected wind turbine blade image data set. Suppose the image data set is ,in is the number of images, for each image , whose pixel coordinates are , , is the number of pixels, , the hidden layer adopts a multi-layer structure, and the number of hidden layers is , No. The number of neurons in the hidden layer is , the formula for determining it is ,in is the adjustment coefficient, with a value range of [0.5, 1.5]. The optimal value under different leaf types and environments is determined through experiments. The number of neurons in the output layer is , corresponding to the recognition result of the leaf tip, define the loss function ,in is the predicted value, is the true value, is a regularization parameter with a value range of [0.001, 0.1], which is used to prevent overfitting. For the Tier The weights of neurons are adjusted through the back propagation algorithm to make the loss function Minimize, thus achieving automatic identification of the blade tip; Step 3: Based on the video tracking algorithm, the position coordinates of the leaf tip in the image are defined as , the image center coordinates are , let the gray value matrix of the image be , whose size is , by calculating the gradient matrix of the image , whose elements are and , use the gradient information to determine the initial position of the leaf tip, and then define a weight matrix based on the gray value distribution characteristics of the pixels around the leaf tip , whose elements It is determined by the distance between the pixel points around the leaf tip and the leaf tip and the gray value difference. Let the distance be , the gray value difference is ,but ,in is the weight adjustment coefficient, the value range is [0.1, 1], and the weighted position coordinates are calculated , in order to achieve high-dynamic, low-latency blade tip tracking and obtain the position of the blade tip in the image in real time; Step 4: Combine the tip height with the high-precision RTK information of the UAV and set the tip height to , the horizontal coordinates in the drone RTK information are , define a space transformation matrix , which is constructed based on the comprehensive consideration of the earth curvature, atmospheric refraction and the UAV's own attitude factors. The earth curvature parameter is , the atmospheric refraction coefficient is , the drone attitude angle is ,but Elements It is derived through complex geometric and physical models, namely , through the space transformation formula ,in The coordinates of the blade tip in the initial reference system are used to calculate the spatial three-dimensional deviation of the blade tip relative to the drone, thereby obtaining the horizontal control amount of the drone. The horizontal position of the drone is controlled through the interface of the drone SDK to make the blade tip as close to the center of the image as possible. At the same time, the drone is controlled to rise in height, so as to achieve approach and contact with the blade tip through visual guidance.

2. The recognition algorithm for the tip of a wind turbine blade by an edge computing platform on a drone according to claim 1 is characterized in that: The installation angle of the flexible copper mesh installation structure and location After precise design, the flexible copper mesh composed of four brackets can adapt to different flight postures and environmental interference while ensuring the stability of the connection with the drone. From the perspective of structural mechanics, the length of the bracket and diameter To satisfy a specific proportional relationship, the proportional coefficients are , then , , , This proportional relationship is obtained through a large number of wind tunnel experiments and simulation analysis, in which different wind speeds are considered. ,wind direction And the flight speed of the drone and posture angle The influence of factors on the stability of the copper mesh, at the same time, the conductivity of the copper mesh material and flexibility To meet specific requirements, the thresholds are set as and , then and This material property can ensure good conductivity and the ability to adapt to the shape of the blade surface when in contact with the blade tip.

3. The recognition algorithm for the tip of a wind turbine blade by an edge computing platform on a drone according to claim 1 is characterized in that: The automation control system adopts a hierarchical control architecture, including the bottom hardware driver layer, the middle control algorithm layer and the upper task scheduling layer. The hardware driver layer is responsible for communicating and controlling the hardware devices of the drone, including motor drive and sensor data acquisition. The control algorithm layer adopts a hybrid control strategy, combining proportional integral differential (PID) control and fuzzy logic control. For the horizontal position control of the drone, the desired position is assumed to be , the current location is , the error is , the parameters of PID control are proportional coefficient , integral coefficient and the differential coefficient , and the determination method is as follows: Assume that the dynamic characteristic parameters of the UAV flight environment are , including wind speed ,wind direction and air density , ,in is a coefficient determined by experiments, with a value range of [0.1, 10]. At the same time, fuzzy logic control is used to deal with uncertainty and nonlinear problems. By defining fuzzy sets and fuzzy rules, the parameters of PID control are dynamically adjusted. The upper task scheduling layer reasonably allocates system resources according to the priority and time requirements of the task to ensure efficient operation of the system.

4. The recognition algorithm of the tip of a wind turbine blade by an edge computing platform on a drone according to claim 1 is characterized in that: The data collected for the deep learning model training data include image data of different types of fan blades under different lighting conditions, different angles and different operating states. For the lighting conditions, there are three conditions: strong light, weak light and natural light. A large amount of image data is collected respectively, and the images are normalized. The normalization function is set as ,in is the original image data, and They are the minimum and maximum values ​​of the data respectively. For the blade angle, it covers all angles from horizontal to vertical. The actual angle of the blade is obtained through precise angle measurement equipment and marked in the image data. For the running state, it includes two situations: stationary and rotating. In the rotating state, different speeds are collected At the same time, in order to improve the generalization ability of the model, the collected data was expanded and processed, and data enhancement technology was used, including rotation, flipping and scaling operations. The probability of each operation Through experiments, it was determined that the value range is [0.1, 1]. This targeted data collection and processing method can improve the accuracy and generalization ability of deep learning models.

5. The recognition algorithm of the tip of a wind turbine blade by an edge computing platform on a drone according to claim 1 is characterized in that: In the implementation process of the video tracking algorithm, a multi-scale feature extraction method is adopted. Assume that the original scale of the image is , through downsampling and upsampling operations, we get image features of different scales, which are For each scale of image features, different feature extractors, namely different layer structures of convolutional neural network (CNN), are used. At the same time, in the tracking process, an adaptive window adjustment strategy is adopted, and the initial tracking window size is set to , according to the movement speed of the leaf tip in the image and direction , dynamically adjust the size and position of the tracking window, and set the adjustment coefficient of the tracking window to , then the window adjustment formula is In addition, in order to improve the tracking accuracy, the color and texture features of the pixels are considered when calculating the weights of the pixels in the window. The color value of the pixel is set to , the texture value is , define the weight function as ,in is the weight adjustment coefficient, It is the reference color value and texture value, determined through experiments.

6. The recognition algorithm for the tip of a wind turbine blade by an edge computing platform on a drone according to claim 1 is characterized in that: The spatial solution formula is reasonable. When deriving the spatial solution formula, the influence of the earth's curvature and atmospheric refraction factors are taken into account. Assuming the radius of the earth is , the atmospheric refraction coefficient is , the horizontal distance of the blade tip relative to the drone is , the height difference is , define a space transformation matrix , which is constructed based on the comprehensive consideration of the earth curvature, atmospheric refraction and the UAV's own attitude factors. The earth curvature parameter is , the atmospheric refraction coefficient is , the drone attitude angle is ,but Elements It is derived through complex geometric and physical models, namely , through the space transformation formula ,in is the coordinate of the blade tip in the initial reference system. At the same time, in order to improve the accuracy of the solution results, in practical applications, the spatial transformation matrix is ​​obtained by real-time acquisition of atmospheric refraction coefficient and earth curvature related data. Dynamic adjustment is performed based on an adaptive algorithm. The adaptive coefficient is set as When the atmospheric refractive index or earth curvature parameter changes exceeding the threshold value, When, yes Elements Update, the update formula is ,in is the change in element calculated based on the new data.

7. The recognition algorithm for the tip of a wind turbine blade by an edge computing platform on a drone according to claim 1 is characterized in that: When using the drone SDK interface to control the horizontal position and height of the drone, the parameters of the interface are carefully set and adjusted. The horizontal control parameters of the interface are set as , the vertical control parameter is Through experiments and simulation analysis, the parameter value range under different control effects is determined. For horizontal position control, when the deviation between the expected position and the current position is within a certain range, the deviation range is set to ,but The value of satisfies ,in and are the lower and upper limits determined by the experiment. For altitude control, when the deviation between the desired altitude and the current altitude is within a certain range, the deviation range is ,but The value of satisfies ,in and are the lower and upper limits determined by the experiment. At the same time, in order to improve the accuracy of control, the flight characteristics of the UAV and the accuracy of the sensor are considered during the control process, and the interface parameters are dynamically adjusted. The adjustment method is based on an adaptive algorithm. The adaptive coefficient is set to When the flight characteristics of the drone or the sensor accuracy changes beyond the threshold, When, yes and Update, the update formula is ,in and is the parameter change calculated according to the new situation.

8. The recognition algorithm for the tip of a wind turbine blade by an edge computing platform on a drone according to claim 1 is characterized in that: The measurement system includes a sensor module and a data processing module. The sensor module uses a high-precision camera and a resistance measurement sensor. The resolution of the camera is ,in and are the number of pixels in the horizontal and vertical directions respectively. By improving the resolution of the camera, a clearer image of the leaf tip can be obtained, which is beneficial for the identification and measurement of the leaf tip. The measurement accuracy of the resistance measurement sensor is By improving the measurement accuracy of the sensor, the resistance value of the blade tip-tower base can be measured more accurately. The data processing module uses a digital signal processing algorithm to filter, amplify and digitize the data collected by the sensor. Suppose the filtering algorithm is , the magnification is , the digital processing function is , then the processed data ,At the same time, in order to improve the accuracy of the ,measurement system, the sensor module and the data processing module were ,coordinately optimized, and the optimal parameter matching relationship between ,the sensor module and the data processing module was determined through ,experiments, including the relationship between the camera resolution and ,filtering algorithm parameters, and the relationship between the resistance ,measurement sensor accuracy and the magnification.

9. The recognition algorithm for the tip of a wind turbine blade by an edge computing platform on a drone according to claim 1 is characterized in that: The blade tip recognition and control process is synergistic. In the whole process, the blade tip recognition, tracking and UAV control are interrelated and synergistic. When the blade tip recognition algorithm detects the position change of the blade tip, it will promptly pass the information to the tracking algorithm and the UAV control algorithm. The tracking algorithm adjusts the tracking strategy according to the position change of the blade tip. The UAV control algorithm adjusts the UAV's speed and altitude according to the tracking results and the spatial solution results. At the same time, the UAV's control state will also be fed back to the blade tip recognition and tracking algorithm so as to timely adjust the algorithm's parameters and strategies. This synergy is achieved through a communication protocol. Suppose the communication protocol is It includes provisions on message format, transmission rate and error handling. That is, the message format specifies the encoding method of blade tip position information and UAV control instructions. The transmission rate is determined according to the flight speed of the UAV and the mission requirements. Error handling specifies how to resend and correct errors when information is transmitted incorrectly. This synergy can ensure the efficient operation and accuracy of the entire system.

10. The recognition algorithm of the tip of a wind turbine blade by an edge computing platform on a drone according to claim 1 is characterized in that: The algorithm as a whole is robust. When facing different environmental conditions, blade types and UAV performance differences, the algorithm can maintain stable performance and accuracy. For environmental conditions, including different wind speeds, wind directions, light and temperatures, by introducing an adaptive mechanism in the algorithm, the algorithm parameters and strategies can be automatically adjusted according to environmental changes. For blade types, wind turbine blades of different models and designs are covered. By extracting and analyzing blade features, it can adapt to the shapes and structures of different blades. For UAV performance differences, the flight characteristics and sensor accuracy differences of UAVs of different brands and models are taken into account. By calibrating and adjusting UAV parameters, the effective operation of the algorithm on different UAVs can be guaranteed.

Citation Information

Patent Citations

  • Fan blade autonomous inspection method based on unmanned aerial vehicle

    CN110554704A

  • Automatic fan blade inspection system and method based on unmanned aerial vehicle

    CN112904877A

  • Wind turbine generator blade multi-type damage detection method and device based on image processing

    CN116402764A

  • Unmanned aerial vehicle visual servo control method and system for fan blade inspection

    CN116912715A

  • Unmanned aerial vehicle load platform for routing inspection of fan blades

    CN118062282A